Progressive Refinement Imaging

TL;DR

This paper introduces a progressive online image integration method that handles large, uncalibrated image sets with geometric and photometric discrepancies, producing consistent high-quality images without global optimization.

Contribution

It proposes a novel progressive refinement technique that avoids global optimization, using a Laplacian pyramid and optical flow for robust image registration and fusion.

Findings

Effective handling of large uncalibrated image sets with discrepancies

Robust image registration using coarse homography and optical flow

High-quality results demonstrated on mobile and consumer camera sequences

Abstract

This paper presents a novel technique for progressive online integration of uncalibrated image sequences with substantial geometric and/or photometric discrepancies into a single, geometrically and photometrically consistent image. Our approach can handle large sets of images, acquired from a nearly planar or infinitely distant scene at different resolutions in object domain and under variable local or global illumination conditions. It allows for efficient user guidance as its progressive nature provides a valid and consistent reconstruction at any moment during the online refinement process. Our approach avoids global optimization techniques, as commonly used in the field of image refinement, and progressively incorporates new imagery into a dynamically extendable and memory-efficient Laplacian pyramid. Our image registration process includes a coarse homography and a local refinement stage using optical flow. Photometric consistency is achieved by retaining the photometric intensities given in a reference image, while it is being refined. Globally blurred imagery and local geometric inconsistencies due to, e.g. motion are detected and removed prior to image fusion. We demonstrate the quality and robustness of our approach using several image and video sequences, including handheld acquisition with mobile phones and zooming sequences with consumer cameras.