Hum You are mixing diferent things here.

Determine your white point. This is not the same as intensity.

Ambient Light and ambient background

Ambient light behind your monitor is a relative thing. Imagine you have a lamp above your desk, but all your things, desk, wall, coffe mug, are painted matt black. The sensor could pick the ambient light as the recoved by the lamp but has nothing to do with the actual background you have behind your monitor.

A quote from here: https://www.just-normlicht.de/en/iso-3664-2009-01.html

Colors in the area surrounding the viewing surface affect perception, which is why the standard prescribes neutral-gray, low-reflection ambient conditions. Be it a booth or room walls, the reflectance of the coating of the viewing environment is set at a maximum of 60%. Unless otherwise stated, JUST standardized viewing booths and stations are coated in neutral gray according to Munsell N7, which meets exactly these requirements.

The problem is that this states for the printed samples, but you could adapt this to your environment behind the monitor. Remove wood finishes (warm color) or distracting elements.

Put a light source of the same temperature of the white point you defined in your workflow.

Adjust the distance of your environmental light so, when projected to the background it gives not more than that 60% compared to the monitor. Measure the reflected light not the incidental light.

(A bright background of about this 60% actually could stress your eyes less than a dark one)

---

One thing you might need to consider is that if you are not using a viewing booth, you might be tempted to compare a printed sample next to your monitor, just try to balance this two situations, try that the viewing light does not cast too much light into your monitor. A shield either the viewing boot or the monitor could be a good option.

---

You can NEVER control a viewing condition on the user's screens. So stop worrying that.

One joke I once comented:

A drunk man is driving on a highway and listen on the radio: "one driver is on the wrong way of the highway" And he says... one? There are a lot of them!

The point of calibrating your monitor is just to be sure you are not the drunk man.