What are the pros and cons of different Bayer demosaicing algorithms?
Asked 4/18/2012
6 views
2 answers
0
Bayer sensors record only one color per photosite, so software must reconstruct full RGB pixels through demosaicing. I understand the simplest methods use nearest-neighbor or basic interpolation, but there are more advanced approaches such as AHD, VNG, PPG, HPHD, and AMaZE.
What do these more advanced demosaicing methods try to improve compared with simpler approaches? What artifacts or tradeoffs are typical with basic methods versus better ones? Also, does limited in-camera processing power tend to force simpler demosaicing for JPEG output compared with desktop raw converters?
Originally by Photography Stack Exchange contributor. Source · Licensed CC BY-SA 4.0
Photography Stack Exchange contributor
14y ago
2 Answers
20
I was surprised a few months ago to find that my machine vision camera SDK used nearest neighbor "interpolation" in its built-in bayer function. It is the fastest, but worst type, giving hard edges, especially when you start doing math on the image channels for colour constancy or brightness invariance. I found this review of algorithms:
http://www.arl.army.mil/arlreports/2010/ARL-TR-5061.pdf
The next step up is bilinear and bicubic interpolations which can be computed quite fast because they amount just to convolution kernels. These give coloured saw-teeth on slanted edges - bilinear more than bicubic.
Can be seen in this paper, and with quantified quality data on 5 different algorithms:
https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/Demosaicing_ICASSP04.pdf
This is why they made edge direction based interpolations. However, these treat green as a more "important channel" (since it has the best resolution and account for most of our visual sensitivity and our eyes' resolution). And then they create blue and red as a function of the green channel, in a hue preserving fashion. This in turn makes green channel high frequency content more prone to errors. Complexity is higher as they have to detect what's going on and require multiple passes. Moire and maizes are common artifacts from these types of interpolation.
Here they show examples of Adaptive Homogeneity Demosaicing and bilinear versions with and without hue preserving and edge preserving addons:
http://math.auburn.edu/~kilgota/ahd_bayer.pdf
That paper favours AHD and doesn't show the negative part. On this page you can see the different pattern artifacts from Adaptive Homogeneity Demosaicing, Patterned Pixel Grouping, and Variable Number of Gradients (hover mouse over the names):
http://www.ruevski.com/rawhistogram/40D_Demosaicing/40D_DemosaicingArtifacts.html
In summary there are a number of assumptions employed in these algorithms, and artifacts occur when the assumption doesn't hold:
- Per channel smoothness. If the nearest neighbors are not the same, make the transition smooth. Artifact: saws/zippers, softness
- Brightness constancy with directed edges (bilinear direction). Artifacts: high frequency texture moire, colour fringing
- Hue constancy. if in a neighborhood the hue is the same, so if one channel changes the others have to follow. Artifacts: colour zippers on colour edges
- Constancy can be predicted from the greens. Artifact: mazes
Originally by user11455. Source · Licensed CC BY-SA 4.0
user11455
13y ago
0
Generated from our catalog & community — verify before relying on it.
Demosaicing methods mainly differ in how well they preserve detail and suppress false color, zippering, and jagged edges.
At the simple end, nearest-neighbor is very fast but usually the poorest quality, producing hard edges and obvious artifacts. Bilinear and bicubic interpolation are also fast because they behave like simple convolution filters, but they can create colored saw-tooth edges and color fringing, especially around high-contrast boundaries and thin near-horizontal/vertical lines.
More advanced methods such as AHD (Adaptive Homogeneity-Directed), VNG (Variable Number of Gradients), and PPG (Patterned Pixel Grouping) try to detect edges and local image structure so they interpolate across smooth areas without blending across edges as much. The goal is better detail retention and fewer Bayer-pattern artifacts or false colors.
Their tradeoff is mostly computational complexity, but also that different algorithms may handle certain scenes better than others; some are preferred specifically when Bayer artifacts appear.
In practice, the “naive” approach is not always terrible, but its weaknesses show up most clearly in high-contrast fine detail. More processing power generally allows more sophisticated demosaicing, which is one reason desktop raw converters often offer multiple algorithms, while in-camera JPEG processing may favor faster, fixed methods.
Recommended products
UniqueBot
AI14y ago