feels wrong to clog up my folders with thousands of xmp files that I'll probably never use

You say "Lightroom CC," by which I assume you also have Photoshop. Every time you say "Edit in Photoshop," you use the XMP data. Photoshop loads it when it loads the photo, and then saves it back out when you save the edited photo.

When you tell Lightroom to "Edit original," this is how Lightroom's changes to the photo get sync'd over into Photoshop. Photoshop will typically end up saving the changes to a different file format than it received the original photo in (PSD or TIFF, most commonly) but the photo metadata still needs to be written out as XMP metadata, else things like camera data get lost in the edit. Despite the edit, you still want the photo marked with your exposure settings, camera model, exposure date, etc.

A number of Lightroom plugins also operate by editing the XMP metadata. If the XMP metadata is out of sync with respect to the Lightroom catalog metadata, Lightroom will put a little arrow badge on the photo to call out the conflict. If you then tell Lightroom to load the metadata from disk, you wipe out any metadata changes in Lightroom's catalog, because Lightroom can't merge the two copies. Alternatively, if you tell Lightroom to overwrite the copy on disk, you lose your plugin's changes to the metadata.

My advice: take the speed hit and leave this option on, always.

Bonus tip: although we have said that writing XMP data takes time, it might not immediately be clear to you how much time. When editing a single photo, it's near-instant. But, if you have a deep and wide keyword hierarchy and change one of the core keywords, such that the XMP metadata for a huge number of photos have to be updated, it can take hours. I tell you this not to talk you out of keeping the "Automatically write changes into XMP" setting turned on, but to point out that there is still a use for Cmd/Ctrl-S with the option turned on: to force Lightroom to save the metadata for a given photo or set of photos immediately when you know you're going to edit it in an app that needs current metadata.

Beware also that Lightroom will pause and resume XMP writes when the app is closed, so if everything isn't written to disk when Lightroom exits, it might resume what it left undone when you start it back up. This is done near-silently in the background; you can't assume that because you restarted Lightroom that all XMP metadata changes are flushed to disk. If you need to be able to see what's going on, select "Metadata Up-to-Date" from the filter bar's preset drop-down menu, then watch the Metadata Status column: if Lightroom is busy writing changes to disk, you'll see the contents of that column changing.

As for the "thousands of files" part, Lightroom only uses separate XMP files with file types that have no way to embed XMP metadata into the file itself. DNG, PSD, TIFF, and JPEG allow embedded XMP metadata. About the only time you see separate XMP files are when you're not letting Lightroom convert your digital camera's raw files into DNGs. Not only does DNG save you from having to keep that separate XMP file associated with the raw photo file, it's probably smaller than your camera's raw file format, without losing much, if anything.