Image sequence interpolation using optimal control

TL;DR

This paper introduces an optimal control framework for image sequence interpolation, modeling images with BV spaces, proving existence of solutions, and demonstrating competitive numerical results against current methods.

Contribution

It formulates image interpolation as an optimal control problem governed by a transport equation, with new theoretical analysis and algorithms.

Findings

The method is competitive with state-of-the-art techniques.

Optimal controls exist and are characterized for the model.

Numerical results outperform several existing interpolation methods.

Abstract

The problem of the generation of an intermediate image between two given images in an image sequence is considered. The problem is formulated as an optimal control problem governed by a transport equation. This approach bears similarities with the Horn \& Schunck method for optical flow calculation but in fact the model is quite different. The images are modelled in $BV$ and an analysis of solutions of transport equations with values in $BV$ is included. Moreover, the existence of optimal controls is proven and necessary conditions are derived. Finally, two algorithms are given and numerical results are compared with existing methods. The new method is competitive with state-of-the-art methods and even outperforms several existing methods.