Supervised Descent Method for Solving Nonlinear Least Squares Problems in Computer Vision

TL;DR

This paper introduces the Supervised Descent Method (SDM), a learning-based approach for efficiently solving nonlinear least squares problems in computer vision without computing derivatives, demonstrating state-of-the-art results in facial feature detection and other tasks.

Contribution

The paper proposes a novel supervised learning framework for descent directions, enabling efficient nonlinear optimization without derivatives, applicable to various computer vision problems.

Findings

SDM achieves state-of-the-art performance in facial feature detection.

SDM converges reliably without computing Jacobians or Hessians.

The method is effective for rigid and non-rigid image alignment and 3D pose estimation.

Abstract

Many computer vision problems (e.g., camera calibration, image alignment, structure from motion) are solved with nonlinear optimization methods. It is generally accepted that second order descent methods are the most robust, fast, and reliable approaches for nonlinear optimization of a general smooth function. However, in the context of computer vision, second order descent methods have two main drawbacks: (1) the function might not be analytically differentiable and numerical approximations are impractical, and (2) the Hessian may be large and not positive definite. To address these issues, this paper proposes generic descent maps, which are average "descent directions" and rescaling factors learned in a supervised fashion. Using generic descent maps, we derive a practical algorithm - Supervised Descent Method (SDM) - for minimizing Nonlinear Least Squares (NLS) problems. During training, SDM learns a sequence of decent maps that minimize the NLS. In testing, SDM minimizes the NLS objective using the learned descent maps without computing the Jacobian or the Hessian. We prove the conditions under which the SDM is guaranteed to converge. We illustrate the effectiveness and accuracy of SDM in three computer vision problems: rigid image alignment, non-rigid image alignment, and 3D pose estimation. In particular, we show how SDM achieves state-of-the-art performance in the problem of facial feature detection. The code has been made available at www.humansensing.cs.cmu.edu/intraface.