Solving Functional Optimization with Deep Networks and Variational Principles

TL;DR

This paper introduces CalVNet, a neural network framework that leverages calculus of variations principles to solve functional optimization problems directly, without training data, applicable to control, filtering, and geometric problems.

Contribution

It presents a novel method that incorporates variational principles into neural network design to solve functional optimization problems without supervision.

Findings

Successfully derives the Kalman filter using first principles.

Solves minimum-time control problems with bang-bang solutions.

Finds geodesics on manifolds without ground-truth data.

Abstract

Can neural networks solve math problems using first a principle alone? This paper shows how to leverage the fundamental theorem of the calculus of variations to design deep neural networks to solve functional optimization without requiring training data (e.g., ground-truth optimal solutions). Our approach is particularly crucial when the solution is a function defined over an unknown interval or support\textemdash such as in minimum-time control problems. By incorporating the necessary conditions satisfied by the optimal function solution, as derived from the calculus of variation, in the design of the deep architecture, CalVNet leverages overparameterized neural networks to learn these optimal functions directly. We validate CalVNet by showing that, without relying on ground-truth data and simply incorporating first principles, it successfully derives the Kalman filter for linear filtering, the bang-bang optimal control for minimum-time problems, and finds geodesics on manifolds. Our results demonstrate that CalVNet can be trained in an unsupervised manner, without relying on ground-truth data, establishing a promising framework for addressing general, potentially unsolved functional optimization problems that still lack analytical solutions.