Total Variation and Euler's Elastica for Supervised Learning

TL;DR

This paper extends total variation and Euler's elastica regularization techniques to supervised learning on high-dimensional data, using radial basis functions to solve the resulting PDEs and demonstrating promising results on benchmark datasets.

Contribution

It introduces a novel approach combining TV and EE regularization with radial basis functions for supervised learning tasks.

Findings

Effective regularization for classification and regression

Promising results on benchmark datasets

Reduction of high-dimensional PDEs to linear coefficient problems

Abstract

In recent years, total variation (TV) and Euler's elastica (EE) have been successfully applied to image processing tasks such as denoising and inpainting. This paper investigates how to extend TV and EE to the supervised learning settings on high dimensional data. The supervised learning problem can be formulated as an energy functional minimization under Tikhonov regularization scheme, where the energy is composed of a squared loss and a total variation smoothing (or Euler's elastica smoothing). Its solution via variational principles leads to an Euler-Lagrange PDE. However, the PDE is always high-dimensional and cannot be directly solved by common methods. Instead, radial basis functions are utilized to approximate the target function, reducing the problem to finding the linear coefficients of basis functions. We apply the proposed methods to supervised learning tasks (including binary classification, multi-class classification, and regression) on benchmark data sets. Extensive experiments have demonstrated promising results of the proposed methods.