OK, after a LOT of googling I finally have an answer:

First of all, I downloaded the facebook template for 360 degree photos, and added the (equirectangular!) spherical photo onto it, and 'saved as' jpg. Here's the Facebook template (edit: new link to template 6000x3000, link to folder). Don't know if it's actually needed, but then you at least have the right metadata, I imagine. Could be wrong though. If you want to post your 360 photo on facebook, you can go right ahead and use this jpg without following additional steps.

NOTE: I had to mirror the image vertically to use it in Google Cardboard. If you don't do this, it will look good on your computer/phone, but when opened in Cardboard Camera, it's mirrored vertically. Don't know if everyone has this, but if you've got the same issue, this is the 'fix' I used. If you only plan to use the photo on facebook, then the mirroring isn't needed.

After saving it, I made a copy of the original, and added "_left" to the original file name and "_right" to the copy file name. The image itself is the same.

Then, I used this web-based application to join the two files. All credits go to Zombieist for this amazing web app.

Save the file generated by Zombieist web app to your phone, and open it in the Cardboard Camera app. Voila, your 360 photo is now shown when using the Google Cardboard!

If anyone has tips/tricks that'll improve this answer, be my guest :D.

NOTE: I have an iPhone, and in the iPhone app you can import vr-images from dropbox and such. Apparently, in the android version (at least on the One+ one) this isn't possible and I haven't found a way to display it in the Android version Cardboard Camera, since it doesn't seem possible to import it. When sharing the link generated by the iPhone Cardboard Camera app, it simply says that it's not a valid VR-format in the android app.