Applications of Automatic Differentiation in Image Registration

TL;DR

This paper explores the use of automatic differentiation (AD) to improve multi-scale affine image registration and super-resolution, demonstrating its effectiveness in computing derivatives crucial for algorithmic enhancements.

Contribution

It introduces novel AD-based methods for image registration and super-resolution, highlighting the importance of exact Hessians and differentiating through projections for improved performance.

Findings

Exact Hessians are necessary for benefits in multi-scale registration.

Differentiating through projections improves super-resolution Jacobian accuracy.

AD enables efficient exploration of complex derivatives in image processing.

Abstract

We demonstrate that automatic differentiation (AD), which has become commonly available in machine learning frameworks, is an efficient way to explore ideas that lead to algorithmic improvement in multi-scale affine image registration and affine super-resolution problems. In our first experiment on multi-scale registration, we implement an ODE predictor-corrector method involving a derivative with respect to the scale parameter and the Hessian of an image registration objective function, both of which would be difficult to compute without AD. Our findings indicate that exact Hessians are necessary for the method to provide any benefits over a traditional multi-scale method; a Gauss-Newton Hessian approximation fails to provide such benefits. In our second experiment, we implement a variable projected Gauss-Newton method for super-resolution and use AD to differentiate through the iteratively computed projection, a method previously unaddressed in the literature. We show that Jacobians obtained without differentiating through the projection are poor approximations to the true Jacobians of the variable projected forward map and explore the performance of other approximations in the problem of super-resolution. By addressing these problems, this work contributes to the application of AD in image registration and sets a precedent for further use of machine learning tools in this field.