On Explicit Curvature Regularization in Deep Generative Models

TL;DR

This paper introduces curvature-based regularization for deep generative models, deriving efficient formulas for intrinsic and extrinsic curvature, and demonstrates their effectiveness over existing methods in autoencoder training.

Contribution

It presents a novel family of curvature regularization terms with explicit formulas and efficient computation for deep generative models, comparing intrinsic and extrinsic curvature effects.

Findings

Curvature regularization improves autoencoder performance.

Intrinsic curvature measures are slightly more effective than extrinsic.

Curvature-based methods outperform existing regularization techniques.

Abstract

We propose a family of curvature-based regularization terms for deep generative model learning. Explicit coordinate-invariant formulas for both intrinsic and extrinsic curvature measures are derived for the case of arbitrary data manifolds embedded in higher-dimensional Euclidean space. Because computing the curvature is a highly computation-intensive process involving the evaluation of second-order derivatives, efficient formulas are derived for approximately evaluating intrinsic and extrinsic curvatures. Comparative studies are conducted that compare the relative efficacy of intrinsic versus extrinsic curvature-based regularization measures, as well as performance comparisons against existing autoencoder training methods. Experiments involving noisy motion capture data confirm that curvature-based methods outperform existing autoencoder regularization methods, with intrinsic curvature measures slightly more effective than extrinsic curvature measures.