Monochrome sensors

At it's core, the imaging sensor in a modern digital camera can be thought of as an array of very tiny solar panels. Solar panels work because photons carry energy. A charge builds up in each "pixel" of the sensor as an analog charge ... but this is converted to a digital value.

This process can be used to explain how a camera produces "an image" ... but not a color image.

Deriving color

To produce color, the camera needs to filter light into separate red, green, and blue values (to mimic the Trichromatic nature of the human eye). (It should be noted that a small percentage of the population (and for reasons of human physiology that I do not understand, this small percentage is always - or nearly always - female) are tetrachromats (four color receptors instead of three). Those who are color-blind are usually dichromats (two colors) ... and just occasionally monochromatic (one color).

The imaging sensor can only record a single value per pixel ... an individual pixel is not sufficient to determine color. A pixel in a full color image has three color channels... a value for red, green, and blue.

There are a few ways to do this.

One (somewhat basic) way is to capture three images (sometimes four). One image is captured with a "red" filter in the image path. This filter blocks wavelengths of light that are not "red" (or nearly red). Another image is captured with a "green" filter. And another with a "blue" filter.

Keep in mind that the visible light spectrum for humans are wavelengths of light from roughly 400 nanometers (a nanometer is 1 billionth of a meter) on the "short" (or high-energy) end of the visible spectrum ... to about 700nm on the "long" (or low-energy) end of the visible spectrum.

In overly-simplistic terms... you can think of "blue" as being light wavelengths between 400-500nm; green as wavelengths from 500-600nm; and red as wavelengths from 600-700nm. Except it's not quite that simple ...

There is considerable overlap between human green and red color cones ... but the blue cone is much farther apart (only a little overlap). Cameras are designed to mimic the sensitivity of human eyes so that the photographs you capture resemble what you recall seeing when you took the photo.

Also... the filters need a bit of overlap in order to record colors that represent blends of the three primary colors.

In pure physics... there is no color ... only a difference of light wavelengths. Color is an interpretation that occurs in our brain. Also, different creatures perceive color differently. Bees, for example, do not have color receptors to detect "reds"... but they DO have receptors that see in the UV range (where human eyes are blind). Dogs are dichromatic and see in blue/yellow.

Color Filter Arrays

It turns out taking a photo with a "blue" filter, followed by another photo taken with a "green" filter, followed by another photo with a "red" filter is a time-consuming process (and very bad for fast-action photography).

So the simpler solution is to employ a color filter array (CFA). Of these, the most common CFA (by far) is the Bayer matrix.

You can think of the Bayer matrix as a filter that resembles a checker-board of colored filters... one over each individual pixel (photo-site) on the camera sensor.

De-Mosaicing to produce three-channel Color

But this presents a new challenge... while the sensor (as a whole) has photo-sites that can detect red or green or blue light... no individual pixel can see all three color channels. But there's a technique (many techniques actually) to solve this.

Each individual photo-site can only record the light that is able to pass through the color filter located directly in front of that individual pixel.

Let's assume it's a "green" pixel ... so it is blind to "red" and "blue" light. How do we give this pixel full-color?

The answer is to inspect the neighboring pixels. E.g. suppose the pixel to your right is "blue"... the pixel to your left will also be blue. Sample the light values recording in each of those "blue" neighbors ... and split the difference. Whatever that value is ... assign it to the "green" pixel's "blue" color channel.

Do the same for red (in the same way that the "green" pixel had two "blue" neighbors... it will also have two "red" neighbors.

It turns out this is a VERY simplistic method to provide full RGB color to each pixel. The process is called "de-Bayering" (if a Bayer filter was used) or "de-Mosaicing" is the generic term when you don't know specifically which color filter array was used). There are multiple de-Mosaicing techniques.