Worst-case Nonlinear Regression with Error Bounds

TL;DR

This paper introduces a novel active-learning approach for nonlinear minimax regression that minimizes worst-case approximation errors with error bounds, using a smooth $L_ abla$ approximation and gradient-based training.

Contribution

It presents a new method combining smooth approximation and global optimization for efficient worst-case nonlinear regression with explicit error bounds.

Findings

Validated on nonlinear functions and control laws.

Achieved efficient training via smooth $L_ abla$ approximation.

Provided a Python library for implementation.

Abstract

We propose an active-learning method for nonlinear minimax regression. Given a nonlinear function that can be arbitrarily evaluated over a compact set, we fit a surrogate model, such as a feedforward neural network, by minimizing the maximum absolute approximation error. To handle the nonsmoothness of this worst-case loss, we introduce a smooth $L_\infty$ approximation that enables efficient gradient-based training. The training set is iteratively enriched by querying points of largest error via global optimization. We also derive constant and input-dependent worst-case error bounds over the entire input domain. The approach is validated on approximations of nonlinear functions and nonconvex sets, uncertain models of nonlinear dynamics, and explicit model predictive control laws. A Python library is available at https://github.com/bemporad/maxfit.