Toward Designing Intelligent PDEs for Computer Vision: An Optimal Control Approach

TL;DR

This paper introduces a data-driven optimal control framework to automatically learn PDEs for various computer vision tasks, simplifying the design process and enabling solutions for previously hard-to-model problems.

Contribution

It proposes a novel PDE-constrained optimal control approach to learn PDEs directly from data, enhancing flexibility and applicability in computer vision.

Findings

Learned PDEs perform well on multiple vision tasks

Able to generate PDEs for previously intractable problems

Framework integrates diverse data sources for training

Abstract

Many computer vision and image processing problems can be posed as solving partial differential equations (PDEs). However, designing PDE system usually requires high mathematical skills and good insight into the problems. In this paper, we consider designing PDEs for various problems arising in computer vision and image processing in a lazy manner: \emph{learning PDEs from real data via data-based optimal control}. We first propose a general intelligent PDE system which holds the basic translational and rotational invariance rule for most vision problems. By introducing a PDE-constrained optimal control framework, it is possible to use the training data resulting from multiple ways (ground truth, results from other methods, and manual results from humans) to learn PDEs for different computer vision tasks. The proposed optimal control based training framework aims at learning a PDE-based regressor to approximate the unknown (and usually nonlinear) mapping of different vision tasks. The experimental results show that the learnt PDEs can solve different vision problems reasonably well. In particular, we can obtain PDEs not only for problems that traditional PDEs work well but also for problems that PDE-based methods have never been tried before, due to the difficulty in describing those problems in a mathematical way.