How can I verify whether image files are corrupted after transfer or download?
Asked 1/14/2014
7 views
2 answers
0
Sometimes a partially transferred image will still open but show only part of the picture, with the rest filled by gray/green blocks or other artifacts. Is there a reliable way to detect this kind of corruption automatically, rather than checking files manually? I'm especially interested in methods for common image formats such as JPEG, and in approaches that can be part of an import or backup workflow.
Originally by Photography Stack Exchange contributor. Source · Licensed CC BY-SA 4.0
Photography Stack Exchange contributor
12y ago
2 Answers
20
If you are talking about JPEG files, then the utility jpeginfo is exactly what you're looking for. It can check files for different types of JPEG errors and corruption and either return an error code (the most useful thing for scripting), or just delete files with errors.
I use this as part of my initial file transfer, to make sure everything copied okay without relying on manual checking. (After that, I make sure their checksums don't change as part of my normal backup/bitrot protection.)
The program is command-line, and comes as source code, but it should be easy to build and use on any Linux distribution or on a Mac with a development environment set up properly. I'm sure you could even do it on Windows with Cygwin or MinGW. (For example, although I can't vouch for its integrity, this blog post seems legit and includes a precompiled download.) To build it yourself:
$ git clone https://github.com/tjko/jpeginfo.git
Cloning into 'jpeginfo'...
[...]
Checking connectivity... done
$ cd jpeginfo/
$ ./configure && make
This should create a jpeginfo command which you can either run in place or copy wherever you want (possibly using make install).
Then, you run it like this:
$ ./jpeginfo -c *.jpg
test1.jpg 1996 x 2554 24bit Exif P 6582168 [OK]
test2.jpg 1996 x 2554 24bit Exif P 6582116 Premature end of JPEG file [WARNING]
test3.jpg Corrupt JPEG data: 1 extraneous bytes before marker 0xe2 1996 x 2554 24bit Exif P 6582169 [WARNING]
Here, test1.jpg is perfectly fine, and test2.jpg I deleted a few bytes from the end, and test3.jpg I changed some random bytes in the header.
If you have RAW files, check out this page from the American Society of Media Photographers on DNG Validation, or one on data validation details, which covers using Adobe's DNG converter to batch-validate proprietary RAW formats. (Unfortunately, this is a GUI operation and not necessarily easily scriptable.)
If you have a camera which natively outputs the 1.2 version of DNG, that's even better, as this includes a built-in MD5 checksum of the image data. Unfortunately, this doesn't seem to be stored with the normal image metadata — or at least exiftool and exiv2 don't recognize it, and they do read 1.2 DNG files in general — which means that as far as I know currently the Adobe validation tool is the only way to take advantage of that too.
Originally by user1943. Source · Licensed CC BY-SA 4.0
user1943
12y ago
0
Generated from our catalog & community — verify before relying on it.
Yes. There are two common approaches:
-
Format-specific validation: For JPEGs, tools such as jpeginfo can parse the file structure and report JPEG errors or corruption. This is useful for detecting truncated or otherwise damaged JPEG files automatically in a script or transfer workflow.
-
Checksums/hashes: For any file type, generate a checksum (for example MD5 or CRC) for the original file and compare it after transfer. This is a standard way to verify integrity for images, RAW files, and sidecars. A common pattern is storing a companion file such as
image.jpg.md5.
Checksums are the more general solution because they work with any file format and can detect later corruption as well. JPEG-specific tools are helpful when you specifically want to validate whether a JPEG file is structurally readable.
So, yes: use a JPEG validator for JPEGs, and use checksums if you want broad file-integrity verification in your workflow.
Recommended products
UniqueBot
AI12y ago
Your Answer
Related Questions
Why do photo files become corrupted on hard drives, and how can I prevent it?
Can Lightroom automatically detect corrupted image files?
How can I identify corrupt RAW files that trigger Lightroom's 'unexpected end of file occurred' error?
Why do some RAW/DNG files show random pink or red block corruption?
How can I find what is corrupting JPEG files on my laptop?