Delaunay-Triangulation-Based Learning with Hessian Total-Variation Regularization

TL;DR

This paper introduces a novel CPWL function learning method based on Delaunay triangulation and Hessian total variation regularization, offering an interpretable and stable alternative to neural networks for regression tasks.

Contribution

It proposes a triangulation-based CPWL learning framework with HTV regularization, providing stability, interpretability, and controlled complexity in regression models.

Findings

Effective in low-dimensional scenarios

Controls model complexity with a single hyperparameter

Validates approach through experimental results

Abstract

Regression is one of the core problems tackled in supervised learning. Rectified linear unit (ReLU) neural networks generate continuous and piecewise-linear (CPWL) mappings and are the state-of-the-art approach for solving regression problems. In this paper, we propose an alternative method that leverages the expressivity of CPWL functions. In contrast to deep neural networks, our CPWL parameterization guarantees stability and is interpretable. Our approach relies on the partitioning of the domain of the CPWL function by a Delaunay triangulation. The function values at the vertices of the triangulation are our learnable parameters and identify the CPWL function uniquely. Formulating the learning scheme as a variational problem, we use the Hessian total variation (HTV) as regularizer to favor CPWL functions with few affine pieces. In this way, we control the complexity of our model through a single hyperparameter. By developing a computational framework to compute the HTV of any CPWL function parameterized by a triangulation, we discretize the learning problem as the generalized least absolute shrinkage and selection operator (LASSO). Our experiments validate the usage of our method in low-dimensional scenarios.