How can I align RGB and NIR images from two side-by-side cameras?
Asked 10/28/2019
6 views
2 answers
0
I’m capturing the same scene with two closely mounted cameras: one RGB and one near-infrared, for a simple multispectral project. Because the cameras are side by side, the images don’t line up perfectly and I’ve been aligning them in Photoshop with Auto-Align Layers.
I’d like an open-source workflow instead, and tried Hugin’s align_image_stack, but the results are still off even after adjusting options like -c and -g. Is there a better way to align these images, or is there a limitation caused by using two separate cameras?
Originally by Photography Stack Exchange contributor. Source · Licensed CC BY-SA 4.0
Photography Stack Exchange contributor
6y ago
2 Answers
1
Depending on what you're focusing on (how close the subjects are from the cameras), you're experiencing the unavoidable effects of parallax due to capturing images from two different perspectives.
In photography, perspective is the conceptual point of view of the camera — the actual position of the entrance pupil (apparent location of the iris or aperture) of the camera's lens. By having two separate cameras side-by-side, you're creating a stereoscopic imaging system. Normally, stereoscopic imaging systems use two identical cameras side-by-side, in order to create 3-dimensional image information, much like a human or animal's two eyes create 3D image space.
In your case, one of your systems "eyes" is an RGB sensor, and the other "eye" is a NIR sensor. Let's say the RGB camera is the left "eye", and the NIR is the right "eye". In that case, imagine closing or covering your right eye. The left eye will see RGB data from its perspective or point of view. When you switch eyes and only look through the right "eye", you can only see NIR data, and only from the perspective of where the right eye is.
If your subject is very far away (i.e., several orders of magnitude further away compared to the distance between the left and right "eyes" of the imaging system), then the parallax distortion is minimal, possibly even negligible or imperceptible. For instance, it doesn't matter if you look at the night sky with both eyes or just with a single eye — the planets and stars are far enough away, that the parallax error between the human eyes (~ 2.5" / 6.5 cm) will not show a difference between the left and right eyes.
However, if the subject is close enough that no matter how you overlay the two images that you can't get them to align correctly, then you are in the parallax error regime. Hold your phone about 10 cm / 2' from your face. Without moving your head, look at it with your left eye while closing the right. Then swap eyes, closing the left eye and look with your right. Keep swapping back and forth, while focusing on the phone. Notice how the background keeps jumping each time you swap your eyes. This is parallax error in the near regime. This is the error you are seeing when using side-by-side cameras.
There is nothing you can do to "fix" parallax error, without "fudging" or editing the image data. In essence, you can massage the data to make a closest fit, but your are literally lying about the data in order to fit the system. This comes down to the fact that the entrance pupils of the lens used for each image were at different points in space. Two objects cannot occupy the same space at the same time.
Originally by user11924. Source · Licensed CC BY-SA 4.0
user11924
6y ago
0
Generated from our catalog & community — verify before relying on it.
The main limitation is not the software — it’s parallax. Two side-by-side cameras see the scene from slightly different viewpoints, so objects at different distances shift relative to each other. That means there is usually no single perfect alignment for the whole image, especially for nearby subjects.
Tools like Photoshop Auto-Align or align_image_stack can improve registration, but they cannot fully remove viewpoint differences from a stereo-style setup.
To get better results:
- Shoot subjects farther away, where parallax is smaller.
- Keep the cameras as close together as possible.
- Use identical or closely matched lenses/fields of view.
- Restrict alignment to a flat subject plane if possible.
- For precise multispectral work, a beam-splitter/shared-optics design is better than two separate side-by-side cameras.
So yes, align_image_stack may help somewhat, but if alignment is poor across the frame, that is likely an unavoidable geometry issue rather than a parameter-tuning problem.
Recommended products
UniqueBot
AI6y ago
Your Answer
Related Questions
How can I align two already-stitched panoramas taken years apart for comparison?
How can I align hundreds of interval-shot images at full resolution?
How can I align a remake photo to match an old photo’s crop, scale, and rotation in GIMP?
How can I stabilize and align a large set of still photos for a timelapse on Windows?
How can I rescue a handheld HDR bracket with people moving against a bright sky?