How can I correct Mars Ingenuity fisheye images with FFmpeg?
Asked 7/15/2021
1 views
2 answers
0
I can already convert a fisheye image to equirectangular with FFmpeg’s v360 filter, but what I really want is to undistort or “rectify” Ingenuity helicopter images so they look more like they were taken with a normal rectilinear camera. NASA provides field-of-view information for the NAV and RTE cameras, but I do not know how to turn that into the right FFmpeg lenscorrection settings. Can FFmpeg do this directly, and if so, which parameters are needed?
Originally by Photography Stack Exchange contributor. Source · Licensed CC BY-SA 4.0
Photography Stack Exchange contributor
4y ago
2 Answers
3
FFmpeg is not the most appropriate tool to use for lens correction of still images. While it may work, it is not really meant to be used that way. Since you are already aware of the documentation, it's a matter of reading and applying it.
lensfun: The lens has to be in the lensfun database. NASA equipment is not typically available to the general public, so they are unlikely to be added to the database.
lenscorrection: You need to calculate k1 and k2, based on this equation:
r_src = r_tgt * (1 + k1 * (r_tgt / r_0)^2 + k2 * (r_tgt / r_0)^4)Other parameters, like cx and cy, can be assumed. There is an OpenCV camera calibration example that calculates the parameters from photos of recognized targets (eg, chessboard pattern). Then you would plug the values into the command:
ffmpeg -i in-file -vf "lenscorrection=cx=0.5:cy=0.5:k1=[]:k2=[]" out-fileUnless NASA has taken and released calibration images, this approach is unlikely to work.
See also:
- Video Production: Correct lens distortion with ffmpeg
- StackOverflow: Is there a way to remove GoPro fisheye using ffmpeg
Other tools to consider:
Hugin: Can calculate and apply lens correction parameters. Cannot be used to find k1 and k2 because it uses a different lens correction model. Will apply generic corrections that may be good enough if lens specifications are known (or guessed).
Photoshop: Adaptive Wide Angle Filter
RawTherapee, and other raw processing applications: Usually include a distortion correction tool with parameters you can tweak for a good-enough approximation.
Note: Some images released by NASA may already be corrected for distortion.

Originally by user75526. Source · Licensed CC BY-SA 4.0
user75526
4y ago
0
Generated from our catalog & community — verify before relying on it.
FFmpeg can do lens correction, but the published field-of-view figures alone are not enough to derive accurate lenscorrection values. For FFmpeg’s lenscorrection filter you need distortion coefficients such as k1 and k2 for the lens model, using a relation like:
r_src = r_tgt * (1 + k1*(r_tgt/r0)^2 + k2*(r_tgt/r0)^4)
You also need the image center (cx, cy), though those can often be assumed near the frame center if unknown.
The main issue is that Ingenuity’s cameras are specialized hardware and are unlikely to exist in the Lensfun database, so lensfun is probably not an option.
In practice, to get usable correction parameters you would normally calibrate the camera from images of a known target (for example with OpenCV camera calibration), then plug the resulting coefficients into FFmpeg. Without those calibration values, FFmpeg cannot infer the correct rectification just from FOV specs.
So: yes, FFmpeg may apply correction, but you first need lens calibration data; the FOV numbers by themselves are insufficient.
Recommended products
UniqueBot
AI4y ago
Your Answer
Related Questions
Why does my 235° fisheye look squeezed when converting to equirectangular in FFmpeg?
How can I defish photos from the Olympus 9mm body-cap fisheye on Linux?
Why can an 11mm fisheye and a 16mm fisheye look similar in field of view?
What software can defish Sigma 8mm fisheye photos into rectilinear images?
Do fisheye lenses focus on a curved surface rather than a flat plane?