How can I automatically pick the sharpest “lucky” landscape frames from a burst?

Asked 10/24/2017

4 views

2 answers

0

I’m photographing a distant landscape subject with telephoto lenses and seeing atmospheric shimmer/turbulence, so some frames in a burst are noticeably sharper than others. Stacking several shots has sometimes looked worse than the best single frame.

Is there a practical way to automate selecting the sharpest images from a set of landscape photos, similar to “lucky imaging” in astronomy? I’m especially interested in open-source or command-line approaches on Linux.

Would measuring image sharpness with something like a Laplacian/contrast metric be a reasonable approach, or is video/high frame rate capture more likely to help in this kind of situation?

Originally by user21068. Source · Licensed CC BY-SA 4.0

user21068

8y ago

2 Answers

6

This is not a definitive answer to my own question, but it's too long for a comment.

I implemented the idea of using the RMS Laplacian. The idea is that if the brightness of the image is represented by an array of pixels a[i,j], then at any point (i,j), we have the discrete approximation to the Laplacian L=a[i-1,j]+a[i+1,j]+a[i,j-1]+a[i,j+1]-4a[i,j]. This measures the sharpness of features in the image. For example, if the image was out of focus, L would be lower. The RMS value of the Laplacian, R, is the square root of the mean of the square of the Laplacian.

Here is my code that calculates R for an input PNG image:

#!/usr/bin/ruby

# To batch convert a bunch of JPGs to png:
# perl -e '$i=0; foreach $f(<*.JPG>) {$s=sprintf("%03d",$i); $c="convert $f $s.png"; print "$c\n"; system($c); $i=$i+1;}'

require 'oily_png'

# require 'hsluv'
  # http://www.hsluv.org
  # https://github.com/hsluv/hsluv-ruby
  # sudo gem install hsluv

# Sloppy and probably not physiologically valid, but fast.
# Returns an integer from 0 to 255*3.
def color_to_brightness(c)
  return ChunkyPNG::Color::r(c)+ChunkyPNG::Color::g(c)+ChunkyPNG::Color::b(c)
end


def rms_laplacian_from_file(input_file)
  image = ChunkyPNG::Image.from_file(input_file)
  n = 0
  sum = 0
  sum_sq = 0
  w = image.width
  h = image.height
  1.upto(w-2) { |i|
    ### if i%1000==0 then print "i=#{i}\n" end # show progress
    next unless i>w/3 && i<(2*w)/3 ## for efficiency, only use center of frame
    1.upto(h-2) { |j|
      next unless j>h/3 && j<(2*h)/3 ## for efficiency, only use center of frame
      next unless rand(10)==0 # for efficiency
      a = Hash.new
      (-1).upto(1) { |k|
        (-1).upto(1) { |l|
          c = image[i+k,j+l] # color, represented as a 4-byte rgba value
          a[[k,l]] = color_to_brightness(c)
        }
      }
      laplacian = a[[1,0]]+a[[-1,0]]+a[[0,1]]+a[[0,-1]]-4*a[[0,0]]
      n = n+1
      sum = sum + laplacian
      sum_sq = sum_sq + laplacian*laplacian
    }
  }
  sum = sum.to_f/n
  sum_sq = sum_sq.to_f/n
  rms = Math::sqrt(sum_sq-sum*sum)
  return rms
end

ARGV.each { |input_file|
  rms = rms_laplacian_from_file(input_file)
  print "#{input_file} -- rms=#{rms}\n"
}

This is implemented in Ruby and runs on Linux using the open-source oily_png library. If anyone's interested in trying it, it should require almost no modification to run on other platforms, if you have Ruby and oily_png installed.

To test that it measures sharpness, I took the first image of my set of 16, measured R, and then added a 5-pixel gaussian blur using GIMP and re-measured R. The result was R=30.8 before the blur, and R=7.8 after. So this does seem to confirm that it measures sharpness.

My 16 images are numbered 000 to 015. Looking at the images by eye, I had previously picked out image 003 as the best. That was the image that I posted a link to in the question.

I ran my code on the 16 shots I had taken, and got the following output:

000.png -- rms=30.809465960392004
001.png -- rms=31.215359700578606
002.png -- rms=31.909926250066476
003.png -- rms=31.83243374839454
004.png -- rms=31.310612756003305
005.png -- rms=30.353258897447564
006.png -- rms=30.61244684985801
007.png -- rms=30.882745734215135
008.png -- rms=28.667104210689384
009.png -- rms=29.862966602367973
010.png -- rms=29.72001987743495
011.png -- rms=30.51274847773823
012.png -- rms=30.84316910530572
013.png -- rms=29.21751498027252
014.png -- rms=29.067434969521976
015.png -- rms=30.831305018709617

Of the 16 images, my choice had the second-highest R value. This would seem to confirm that this statistic could be a useful as an alternative to inspecting images and judging them subjectively by eye.

My implementation is pretty slow, and to make up for that I did some things to improve its performance. I only inspect the middle of the field, and I only sample the Laplacian at 1/10 of the points. In a more optimized implementation, these shortcuts could be eliminated if desired.

It later occurred to me that there might be a much simpler way to do this. An image with more detail should not compress as well, so the largest JPG file might simply be the best one. Sure enough, doing an ls -lS to list the files in order of decreasing size gave a list that was very nearly in the same order as the files sorted by decreasing R:

-rw-rw-r-- 1 bcrowell bcrowell 16970354 Oct 25 15:48 003.png
-rw-rw-r-- 1 bcrowell bcrowell 16927174 Oct 25 15:48 002.png
-rw-rw-r-- 1 bcrowell bcrowell 16903104 Oct 25 15:48 004.png
-rw-rw-r-- 1 bcrowell bcrowell 16882373 Oct 25 15:47 000.png
-rw-rw-r-- 1 bcrowell bcrowell 16861082 Oct 25 15:47 001.png
-rw-rw-r-- 1 bcrowell bcrowell 16817527 Oct 25 15:48 006.png
-rw-rw-r-- 1 bcrowell bcrowell 16816529 Oct 25 15:49 011.png
-rw-rw-r-- 1 bcrowell bcrowell 16793982 Oct 25 15:49 012.png
-rw-rw-r-- 1 bcrowell bcrowell 16786443 Oct 25 15:48 009.png
-rw-rw-r-- 1 bcrowell bcrowell 16773575 Oct 25 15:48 005.png
-rw-rw-r-- 1 bcrowell bcrowell 16771759 Oct 25 15:49 010.png
-rw-rw-r-- 1 bcrowell bcrowell 16765674 Oct 25 15:48 007.png
-rw-rw-r-- 1 bcrowell bcrowell 16764562 Oct 25 15:49 015.png
-rw-rw-r-- 1 bcrowell bcrowell 16750179 Oct 25 15:48 008.png
-rw-rw-r-- 1 bcrowell bcrowell 16732854 Oct 25 15:49 013.png
-rw-rw-r-- 1 bcrowell bcrowell 16684073 Oct 25 15:49 014.png

Originally by user21068. Source · Licensed CC BY-SA 4.0

user21068

8y ago

0

AI Answer

Generated from our catalog & community — verify before relying on it.

Yes. If your goal is to find the sharpest frame in a burst, you can automate it by scoring each image with a sharpness metric and sorting by that score.

A reasonable method from the answers is the RMS Laplacian: compute the image Laplacian (a measure of local intensity change) and then take its RMS value. Sharper images usually produce stronger high-frequency detail and a higher score, while blur lowers it. This is closely related to how autofocus systems evaluate sharpness/contrast.

Caveat: Laplacian-based measures also amplify noise, so they are not perfect. In low-contrast scenes or noisy files, other contrast/focus measures may perform better, and this remains an active area of research. An edge-based metric could also be explored, but that was suggested only as a possible alternative.

So the practical answer is: yes, automated selection is feasible, and a Laplacian/contrast-based ranking is a sensible starting point for landscape “lucky imaging.” The provided answers do not establish that video is better; they mainly support frame selection using image-sharpness metrics.

UniqueBot

AI

8y ago

Your Answer