Overestimation learning with guarantees

TL;DR

This paper introduces a method to train neural networks that are guaranteed to overestimate a reference function, useful for resource-constrained embedded systems, by constructing adaptive overestimators with theoretical guarantees.

Contribution

The paper presents a novel two-step method combining adaptive point selection and neural network optimization to ensure overestimation guarantees under monotonicity assumptions.

Findings

The method guarantees overestimation of the reference function.

Majoring points concentrate where the function varies most.

Surrogate functions provide good approximations in experiments.

Abstract

We describe a complete method that learns a neural network which is guaranteed to overestimate a reference function on a given domain. The neural network can then be used as a surrogate for the reference function. The method involves two steps. In the first step, we construct an adaptive set of Majoring Points. In the second step, we optimize a well-chosen neural network to overestimate the Majoring Points. In order to extend the guarantee on the Majoring Points to the whole domain, we necessarily have to make an assumption on the reference function. In this study, we assume that the reference function is monotonic. We provide experiments on synthetic and real problems. The experiments show that the density of the Majoring Points concentrate where the reference function varies. The learned over-estimations are both guaranteed to overestimate the reference function and are proven empirically to provide good approximations of it. Experiments on real data show that the method makes it possible to use the surrogate function in embedded systems for which an underestimation is critical; when computing the reference function requires too many resources.