# Robust Alignment for Panoramic Stitching via an Exact Rank Constraint

## TL;DR

This paper introduces a pixel-based, globally-aware image alignment algorithm for panoramic stitching, formulated as a rank-1 and sparse matrix decomposition problem, with proven recovery guarantees and broad applicability.

## Contribution

It presents a novel non-convex optimization approach for pixel-level alignment that generalizes to multiple images and homographies, outperforming existing methods.

## Key findings

- More accurate image alignment than state-of-the-art methods
- Higher quality stitched panoramic images achieved
- Efficient algorithm with proven convergence and recovery guarantees

## Abstract

We study the problem of image alignment for panoramic stitching. Unlike most existing approaches that are feature-based, our algorithm works on pixels directly, and accounts for errors across the whole images globally. Technically, we formulate the alignment problem as rank-1 and sparse matrix decomposition over transformed images, and develop an efficient algorithm for solving this challenging non-convex optimization problem. The algorithm reduces to solving a sequence of subproblems, where we analytically establish exact recovery conditions, convergence and optimality, together with convergence rate and complexity. We generalize it to simultaneously align multiple images and recover multiple homographies, extending its application scope towards vast majority of practical scenarios. Experimental results demonstrate that the proposed algorithm is capable of more accurately aligning the images and generating higher quality stitched images than state-of-the-art methods.