Prevention of Overfitting on Mesh-Structured Data Regressions with a Modified Laplace Operator

TL;DR

This paper introduces a novel method to prevent overfitting in mesh-structured data regressions by using a modified Laplace operator to detect oscillations and optimize hyperparameters without data splitting.

Contribution

The work presents a new approach employing a modified Laplace operator on mesh data for overfitting detection and hyperparameter tuning, avoiding traditional data splitting.

Findings

Reduced overfitting through entropy minimization of the Laplace derivatives

Effective detection of oscillations in mesh data

Hyperparameter optimization without data splitting

Abstract

This document reports on a method for detecting and preventing overfitting on data regressions, herein applied to mesh-like data structures. The mesh structure allows for the straightforward computation of the Laplace-operator second-order derivatives in a finite-difference fashion for noiseless data. Derivatives of the training data are computed on the original training mesh to serve as a true label of the entropy of the training data. Derivatives of the trained data are computed on a staggered mesh to identify oscillations in the interior of the original training mesh cells. The loss of the Laplace-operator derivatives is used for hyperparameter optimisation, achieving a reduction of unwanted oscillation through the minimisation of the entropy of the trained model. In this setup, testing does not require the splitting of points from the training data, and training is thus directly performed on all available training points. The Laplace operator applied to the trained data on a staggered mesh serves as a surrogate testing metric based on diffusion properties.