Read noise is essentially circuit noise from readout of the sensor photosites up through ADC conversion into digital values. It is typically mostly amplifier and capacitor switching noise.

But that is an input referred noise graph. It does not mean what you think it does... more the opposite really. It is a graph of the input current required to generate the same system noise as the circuit does in normal use.

What you probably want is this chart: "Read Noise in DNs" (digital numbers)... which includes ADC error/noise contributions. https://photonstophotos.net/Charts/RN_ADU.htm#Nikon%20Z%206II_14

Here you see the same drop in read noise at ISO 800; but otherwise noise increases with ISO, and the lowest value is at base ISO as one would expect.

That drop in read noise is typical of a sensor that uses dual gain photosites, and it occurs when the second gain stage (capacitor) is disabled. In this case it does indeed mean you would be better off using ISO 800 than ISO 318-640 in terms of read noise.

But probably equally relevant is this chart: "Photographic Dynamic Range Shadow Improvement" https://photonstophotos.net/Charts/PDR_Shadow.htm

This chart shows the increase in useful information recorded by using a higher ISO (as compared to underexposing/recovering). You can see that from base ISO - 640 there is only about .25 stop/EV of improvement (less than ~.5 stop is visually insignificant). But at ISO 800 there is a full stop increase, which is significant. And then it essentially doesn't change after that point; which means the sensor is "ISO Invariant" from 800 on. Basically the camera has (effectively) two levels of ISO invariance... ISO 100 for dual gain operation, and ISO 800 for single gain operation.

Edit: I do have permission to reuse/repost Bill's copyrighted material in context.