Note: This definitely is not an all-around solution, but it might be a start.

When I re-read your question, I had a quick look into my PSD-files, only to find out that they lack thumbnails, too.

It seems that Microsoft has dropped the support for PSD-thumbnails, and Adobe seemingly does not care about it.

One solution I found is SageThumbs - it is free and open source. I tried it just now and it seems to work. But: Layered PSDs show a white thumbnail. If I open the PSD in question in IrfanView, it is white, too - in Photoshop, it is not, of course.

Maybe, XnView's plug-ins could cure that, as SageThumb's main website says:

SageThumbs can use external XnView plugins, it tries to automatically detect XnView installation folder or if failed it will use predefined folder.

Typically this folder is C:\Program files\XnView\PlugIns\ for 32-bit SageThumbs on 32-bit Windows or 64-bit SageThumbs on 64-bit Windows and C:\Program Files (x86)\XnView\!PlugIns\ for 32-bit SageThumbs on 64-bit Windows.

So if you have no XnView installed you can create this folder manually and unpack plugins to it.

I do not want to install XnView or its plug-ins to test this right now, so I cannot prove/refute that this would help with layered PSDs.

---

Also, SageThumbs seems to support CR2-files that Windows does not support (EOS M6, for example) - however, they all have a purple-shift.

---

Not really an answer: If this really bugs you but you do not like SageThumbs, you could still use Adobe Bridge instead of the File Explorer. In Bridge, thumbs come up as expected, though it really does not solve the question of thumbnails in Windows' File Explorer.