How can I fairly test and compare camera noise at different ISO settings?

You can measure the noise ratio at different ISO settings in a fairly simple way and use Gimp (Photoshop) to visualise the results. Below are the results with my camera, a Pentax K7.

ISO 100 is used as the basis for comparison.
The Noise Ratio is a dimensionless number showing the increased noise above that of ISO 100.

If you photograph a perfectly uniform grey surface, the standard deviation of the pixel values should be exactly zero. Any deviation from that should be noise of different types.

But, in practice we can't create a perfectly uniform grey surface. So we photograph the same surface at all ISO settings and use the ratio of increased standard deviation above that at ISO 100 to measure the ratio of increased noise.

One can see below, that my K7, for all its virtues, is not a stellar noise performer.

Procedure

Take test photos as follows:

• photograph a uniform dark surface at each ISO setting as follows.
• set the aperture to the maximum.
• focus on infinity, we want the image as blurred as possible.
• determine the normal exposure and decrease the shutter speed by 2 EV.
• mount the camera on a tripod, each photo must cover exactly the same area.
• set your camera to raw.

Process the test photos as follows:

• set the raw convertor to no profile, linear input, linear output, default exposure (not auto).
• open the result in Gimp and select the central 500 x 500 pixel area. It is important to select the same area in each photo.
• Select Colors|Info|Histogram and note the mean and standard deviation readings.
• Calculate the coefficient of variation = (std. dev.) / mean.
• Calculate the Noise Ratio = (coefficient of variation of test sample) / (coefficient of variation at ISO 100).

Results for Pentax K7

To help visualise the results I show the extracted noise at each ISO by by overlaying the test layer over the base layer at ISO 100 and setting the layer mode to Grain Extract. This is used as the background of the histogram, below. Note how the histogram spreads out as the noise increases. Performance up to ISO 400 is good but noises creeps in from ISO 800 onwards.

ISO 100 standard deviation, the background shows extracted noise.

ISO 200 standard deviation, the background shows extracted noise.

ISO 400 standard deviation, the background shows extracted noise.

ISO 800 standard deviation, the background shows extracted noise.

ISO 1600 standard deviation, the background shows extracted noise.

ISO 3200 standard deviation, the background shows extracted noise.

ISO 6400 standard deviation, the background shows extracted noise.