How useful are DxOMark scores for comparing cameras?
Asked 2/24/2014
5 views
2 answers
0
DxOMark publishes lab measurements and overall scores for camera sensors. Sometimes very different cameras receive similar overall scores, which makes the rankings seem questionable. How should photographers interpret DxOMark results? Are the individual measurements more useful than the overall score, and what are the limits of using DxOMark to compare cameras across brands or in real-world shooting?
Originally by Photography Stack Exchange contributor. Source · Licensed CC BY-SA 4.0
Photography Stack Exchange contributor
12y ago
2 Answers
20
DXOMark primary "scores" are utterly useless. IGNORE THEM. It is a futile effort to try and reduce a complex entity such as a DSLR to a single, scalar number that tells you everything about it. It's a fallacy. There are too many factors to consider, and which factors are most important for a given photographer differ. A single score entirely defeats the purpose of running measurements in the first place.
When it comes to DXO's other scores, such as low light and landscape and whatnot, take them with a hefty dose of salt. Their general scores are heavily weighted, and often based on derivations of measurements rather than actual measurements. For example, the Landscape score is based on the Print DR "measure". The problem is that DXO does not actually MEASURE Print DR, in that it is not based on samples taken from actual downsampled images. Print DR is a simple mathematical extrapolation from the TRUE measured dynamic range of the sensor.
Therefor, Print DR does not really tell you anything about the sensor. When DXO says the D800 and D600 have 14.4 stops of DR, that is Print DR, which is extrapolated from the actual hardware DR, which is 13.2 stops. Same thing goes for Canon sensors. When DXO tells you the sensor has 12 stops of DR, that really isn't the case. In reality, most Canon sensors, at a hardware level, have around 10.95 stops of DR.
The problem is worse than this, however. Much of the color depth and color sensitivity scoring information is weighted, as are many of the ISO-based scores. Weights are based on cameras achieving certain thresholds, such as the SNR at a certain ISO being higher than a certain level. This grants a certain "bonus" to the score for that camera. The moment any kind of weight-based bonus scoring enters into the game, your ability to directly compare anything by any score completely goes out the window. Your now on a non-linear playing field where you honestly don't know if that Nikon camera over there with a score of 95 has been heavily weighted relative to this Canon camera over here with a score of 80.
When it comes to actual measurements, DXO information is some of the best available. Their measures of SNR, Screen DR, color sensitivity, etc. are quite sound, as it's all taken directly from multiple RAW images samples for each camera tested. Their testing methodology is fairly rigorous, and there is nothing to indicate that part of their methodology should be doubted. Scientifically, as far as how they test and what they measure, DXO has solid practices and solid information.
DXO is really a mixed bag. They may have solid testing practices, but their scoring, given that it is often based on mathematically derived, weighted information and the fact that a couple of their scores are often given "bonus" points simply for meeting certain thresholds, completely debases the entire point of what DXO does: To produce a linear score for each camera that allows cameras to be easily compared. It was a flawed concept to start with, single-number scoring, but they made it so much worse by how they handle the actual scoring process.
Originally by user124. Source · Licensed CC BY-SA 4.0
user124
12y ago
0
Generated from our catalog & community — verify before relying on it.
DxOMark can be useful, but mostly for its individual lab measurements—not the headline score.
The main problem with the overall score is that it compresses several different characteristics into one number. That makes it easy to rank cameras, but not very meaningful, because photographers value dynamic range, color depth, high-ISO performance, speed, autofocus, ergonomics, and many other factors differently. A single score can make very different cameras look “equal” when they are not equivalent in real use.
Their sub-scores and measurements can still be informative if you understand what they represent and their limits. They are lab-based results, not a full prediction of real-world performance. That means they may be somewhat useful for narrow comparisons—especially when looking at similar cameras—but should be treated cautiously, particularly across different brands or camera types.
Best practice: use DxOMark as one data point. Read the underlying measurements, ignore the overall ranking, and compare those results with real-world reviews, sample images, and your own shooting needs.
Recommended products
UniqueBot
AI12y ago
Your Answer
Related Questions
Do Nikon DSLRs really have better sensors than Canon, and how useful are DxOMark scores?
Can DxOMark lens scores be compared across camera systems?
Should I rely on SnapSort scores when choosing between a Canon T5i and Nikon D5300?
Why can a lens get weak lab reviews but still have many happy owners?
How can I compare different cameras with simple test shots?