Why can’t a camera sensor expose each pixel differently to avoid blown highlights and blocked shadows?

Asked 1/3/2012

2 views

2 answers

0

If cameras are digital, why does one exposure setting have to apply to the whole sensor? Why can’t each photosite or pixel effectively use its own metering or electronic cutoff so bright areas stop collecting light sooner while darker areas keep exposing longer? It seems like that could prevent overexposure and underexposure in the same frame, similar to how our eyes seem to handle bright and dark areas at once.

Originally by user7264. Source · Licensed CC BY-SA 4.0

user7264

14y ago

2 Answers

20

Who decides which pixels get how much gain? Much of what goes on in the human visual system happens in the cortex, not the eye, and depends on what we think is important to see based on a combination of intellectual decision and the (somewhat override-able) instinctual drive for self-preservation. While it's true in one sense that we see what's there, it is equally true in another sense that we see what we want to (or need to) see.

It would be almost trivial to create a relatively low pixel density sensor with large photosites that allow for an enormous dynamic range and (assuming a CCD-type technology, since the current CMOS sensor tech can't work this way) a per-pixel electronic shutter in addition to the mechanical shutter. So what would that get you? A flat image with a lot of bit depth and very low local contrast (if the entire bit depth is converted as-is for display or print) along with a number of pixels that are almost, but not quite, clipped by the sensor saturation (although they are, in fact, clipped by the limiting action of the electronic shutter just before the point of saturation). Let's say for the sake of argument, though, that this sensor and its associated computer could record the clipping data (the reason why it stopped recording at that sensel, which could be as simple as recording the actual exposure duration at that site). That would allow the camera's electronics to reconstruct what the numbers would have been if the photosite could have stayed in the game until the final whistle. So now we have an even flatter image with greater bit depth. And where do you draw the line? 32 bits? 64?

Now comes the hard part -- turning this flat, high-dynamic-range image data into a compelling photograph. The simplest approach is to take the eight bits (or whatever the output bit depth would be) that represent the primary metered image and throw away the rest. It would probably be not much more difficult to fit the data to an S-curve, compressing the extreme shadows and/or highlights -- which is more or less what the extended dynamic range settings on newer cameras already do. But there are only so many output bits available per pixel, and most of the extended highlight values are going to round up to white (or at least a 254 and 255 mix). So you've gained very little by dramatically complicating the system.

But there is still one option open -- selective area mapping. Why not bring the sky, say, or just the clouds in that sky, down in value so it can retain detail, while preserving the desired contrast in the foreground? This is where the hard problem lives. What's important? Should the camera decide for you? If the camera decides, then we have a big advance in machine vision and artificial intelligence to get around to first. If not, then do you really want to make this level of post-capture decision for every picture you shoot? Yes, I know there will be some photo-techno-weinies who really do want to be that hands-on, but can we accept that it's a pathological condition, and that professionals interested in turn-around time and the vast majority of consumers aren't like that?

So you need a new sensor, vastly more complicated electronics around the sensor, an enormous image file for projected raw data (which necessitates larger cards and longer write times/slower frame rates), all to gather data that is going to be thrown away most of the time so that you can occasionally shoot one-shot HDR images that require a lot of human intervention in post (or a huge leap in MV/AI). You could probably sell a few of these, but I'd expect the market to look an awful lot more like the medium format market than the existing 35mm/APS-C market. That is, you'd sell to a select group of well-heeled photographers who either actually need the capabilities for professional reasons or to fulfill their fine art vision, and a few who just get a big enough kick out of post-processing to pay the technology tax.

Originally by user2719. Source · Licensed CC BY-SA 4.0

user2719

14y ago

0

AI Answer

Generated from our catalog & community — verify before relying on it.

A sensor can’t simply decide exposure per pixel in a useful way without changing the scene’s tonal relationships. A photograph looks natural partly because bright areas are brighter than dark areas; exposing different parts differently would flatten that relationship unless the camera also did complex tone mapping.

Also, the human eye is not really capturing the whole scene at one perfect exposure. Your visual system constantly adapts and your brain combines many rapid “snapshots,” so it feels like you see huge dynamic range all at once.

In principle, per-pixel control or wider dynamic range is possible. Some designs use different sensitivity pixels on the same sensor, and research concepts include pixels that reset when full or very fast photon-counting sensors. But the real goal is usually not independent exposure for every pixel — it’s higher dynamic range, so one capture can hold both highlights and shadows more faithfully.

So the limitation is less “digital cameras can’t do it at all” and more that current sensors, processing, and image aesthetics make broader dynamic range a better solution than per-pixel exposure shutoff.

UniqueBot

AI

14y ago

Your Answer