The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences

TL;DR

This paper introduces the Projected Power Method, an efficient algorithm for joint alignment from pairwise differences that converges to the maximum likelihood estimate under broad conditions, with demonstrated effectiveness on synthetic and real data.

Contribution

The paper proposes a low-complexity, model-free projected power iteration algorithm for discrete joint alignment problems, with theoretical convergence guarantees.

Findings

Algorithm converges to the maximum likelihood estimate in broad models.

Effective on both synthetic and real datasets.

Operates efficiently in a lifted orthogonal space.

Abstract

Various applications involve assigning discrete label values to a collection of objects based on some pairwise noisy data. Due to the discrete---and hence nonconvex---structure of the problem, computing the optimal assignment (e.g.~maximum likelihood assignment) becomes intractable at first sight. This paper makes progress towards efficient computation by focusing on a concrete joint alignment problem---that is, the problem of recovering $n$ discrete variables $x_i \in \{1,\cdots, m\}$, $1\leq i\leq n$ given noisy observations of their modulo differences $\{x_i - x_j~\mathsf{mod}~m\}$. We propose a low-complexity and model-free procedure, which operates in a lifted space by representing distinct label values in orthogonal directions, and which attempts to optimize quadratic functions over hypercubes. Starting with a first guess computed via a spectral method, the algorithm successively refines the iterates via projected power iterations. We prove that for a broad class of statistical models, the proposed projected power method makes no error---and hence converges to the maximum likelihood estimate---in a suitable regime. Numerical experiments have been carried out on both synthetic and real data to demonstrate the practicality of our algorithm. We expect this algorithmic framework to be effective for a broad range of discrete assignment problems.