Learning Globally Smooth Functions on Manifolds

TL;DR

This paper introduces a novel method combining semi-infinite constrained learning and manifold regularization to learn globally smooth functions on manifolds, improving generalization and stability in learning tasks.

Contribution

It establishes the equivalence between Lipschitz function learning and a weighted manifold regularization, leading to a practical algorithm with adaptive weights.

Findings

The method effectively estimates the Lipschitz constant.

Experiments show improved performance over existing methods.

The approach ensures globally smooth solutions on manifolds.

Abstract

Smoothness and low dimensional structures play central roles in improving generalization and stability in learning and statistics. This work combines techniques from semi-infinite constrained learning and manifold regularization to learn representations that are globally smooth on a manifold. To do so, it shows that under typical conditions the problem of learning a Lipschitz continuous function on a manifold is equivalent to a dynamically weighted manifold regularization problem. This observation leads to a practical algorithm based on a weighted Laplacian penalty whose weights are adapted using stochastic gradient techniques. It is shown that under mild conditions, this method estimates the Lipschitz constant of the solution, learning a globally smooth solution as a byproduct. Experiments on real world data illustrate the advantages of the proposed method relative to existing alternatives.