I think there's a broad misconception here, but I'm not sure where to start to 'fix' that, so here are some rambling thoughts so far…
Bullet points in the question have been edited since I answered this, so my numbers no longer match the question. The broad scope is still the same.

1. I tested the images from the linked website in four browsers, Left to right in the screenshot below, Safari & Chrome on Mac, Edge & Chrome on Windows. They all look pretty close. Both computers are colour profiled, though my Windows computer is never used for anything mission-critical so it's likely to have drifted over time. The Windows half of the screenshot is also over RDC, so that's another potential pitfall. Allowing for that, I think they're 'close enough'.

Click for full size

Also, both images taken from the site & opened in Photoshop look identical. The sRGB picture is actually an untagged RGB, so will be assumed to be sRGB. The ProPhoto seems to contain the correct profile.
That they appear identical is to be expected on a calibrated machine using an image originally from an sRGB source which I can only guess that was. My displays can display Adobe RGB but not full ProPhoto, so I wouldn't be able to see what may have been lost outside that gamut.

2. This all depends on how you "change" the profile…

3. Convert to profile will attempt to preserve a "visual match", even though ProPhoto can encompass a larger gamut [actually larger than any monitor can display or any eye can see].
   Assign profile never works visually, because it doesn't attempt any compensation. It would be more useful to re-attach a known profile lost from an image.
   Example of assigning sRGB to a ProPhoto image - colour is completely borked.
   
   This may indeed be similar to the description from the linked site - "[If it] appears to have a dark blue/magenta sky with yellowish sand, then your browser is not color-managed."

4. Once you have done your first conversion down from ProPhoto to sRGB, you ought then to be able to cycle round from one to the other seeing no change [you might get drift over time, this is not something I've ever needed to test].

So as this all ought to work in theory… we have to try figure out where you went wrong.
That you see different colours in Photoshop tells me Ps is set up wrongly.
The most common mistake is that people assign their screen icc profile as their working space.
Secondly, that on import, automatic profile conversion is done… to that wrong working space.
This leads to multiple erroneous compensations, one in Ps, the other as the image is displayed to your screen

You first need to make sure you don't have this set up in your workflow.

Aside from that, all this is going to be far more accurate if your screen is first accurately profiled - otherwise everything is just guesswork. Just dialling in the manufacturer's profile is not accurate enough for colour work.
However, no matter whether your profiling is correct or not, having it correctly assigned in your workflow should prevent this apparent drift between images.