The first thing to tackle is keeping all your focusing screens (and equipment used to handle them) tidy. Any dirt will be clearly visible in viewfinder and somewhat hard to remove from the ground side of the glass. Often a focusing screen kit has tweezers and a finger-protection glove (like a tiny condom, to protect the glass from body grease) to assist handling the focusing screens.

I'd also worry about wearing off the clip holding the focusing screen in place - it's not designed for frequent fiddling, so I'd expect the screen to become rickety or not lock in place any more after frequent switching.

Third, changing focusing screens involves fiddling with small items using clumsy tools in a tight space and becomes quite tedious.

I kept the original glass after changing mine in a drawer "just in case", but have never bothered switching back. Although the replacement is slightly dimmer, the image pops more clearly when in focus. And even when the middle prisms become black with a consumer zoom, I'm likely more interested in my subject (usually off-center) and edges.