Least Squares Normalized Cross Correlation

TL;DR

This paper introduces a straightforward, efficient least squares approach to normalized cross correlation (NCC) for image alignment, effectively handling local intensity variations and occlusions, and outperforming existing methods in convergence and speed.

Contribution

It demonstrates that a direct least squares optimization of NCC is feasible and introduces a robust, locally normalized formulation with sparse features for improved efficiency.

Findings

Improved convergence rate over existing lighting invariant methods

Reduced computation time in image alignment tasks

Enhanced robustness to local intensity variations and occlusions

Abstract

Direct methods are widely used for alignment of models to images, due to their accuracy, since they minimize errors in the domain of measurement noise. They have leveraged least squares minimizations, for simple, efficient, variational optimization, since the seminal 1981 work of Lucas & Kanade, and normalized cross correlation (NCC), for robustness to intensity variations, since at least 1972. Despite the complementary benefits of these two well known methods, they have not been effectively combined to address local variations in intensity. Many ad-hoc NCC frameworks, sub-optimal least squares methods and image transformation approaches have thus been proposed instead, each with their own limitations. This work shows that a least squares optimization of NCC without approximation is not only possible, but straightforward and efficient. A robust, locally normalized formulation is introduced to mitigate local intensity variations and partial occlusions. Finally, sparse features with oriented patches are proposed for further efficiency. The resulting framework is simple to implement, computationally efficient and robust to local intensity variations. It is evaluated on the image alignment problem, showing improvements in both convergence rate and computation time over existing lighting invariant methods.