The JPEG compression algorithm works like this:

• Instead of being saved as three R, G and B planes, your image is decomposed into 3 planes, one that carries luminosity, and two that carry color (aka chroma) information.
• The next step is called "chroma sub-sampling": the two chroma planes can then be scaled down by a factor of two in one dimension (chroma halved), or by a factor of two in both dimensions (chroma quartered)*.
• The lossy compression is then applied to each plane.

So if you start with an image with N pixels, you initially have:

N + N + N = 3×N 

pixels to save. But with the chroma halved you have:

N + ½N + ½N = 2×N 

pixels, and with the chroma quartered your have:

1 + ¼N + ¼N = 1.5×N 

So, when the chroma is quartered you have reduced the size of the data to a half before even applying the compression. This directly results in a much smaller file size.

Some applications have independent settings for quality and chroma, while others use a given subsampling depending on the quality setting.

---

*This works because our eyes are much more sensitive to luminosity than to color. Try this (in Photoshop or Gimp):

• Take an image
• Make one copy B&W
• In the other copy:
  • Scale it down by 2 in both directions (you can even try a factor of 4)
  • Scale it up to its original size. In effect you have blurred your image
• Import the color version as a new layer
• Set it to "Color" blend mode (so the resulting image is the color of the top layer applied to the luminosity of the bottom layer)

You won't see much difference with the initial image.