Precomputable Trade-off Between Error and Breakpoints in Piecewise Linearization for First-Order Loss Functions

TL;DR

This paper develops a precomputable upper bound on the number of breakpoints needed for piecewise linear approximation of first-order loss functions, balancing error and computational complexity in stochastic optimization.

Contribution

It introduces a precomputable upper bound on breakpoints for piecewise linearization, aiding efficient approximation with controlled error in stochastic optimization.

Findings

Derived an upper bound on breakpoints for a given error

Proposed algorithms to achieve approximations below the upper bound

Facilitates preemptive trade-off decisions in model approximation

Abstract

Stochastic optimization often involves calculating the expected value of a first-order max or min function, known as a first-order loss function. In this context, loss functions are frequently approximated using piecewise linear functions. Determining the approximation error and the number of breakpoints (segments) becomes a critical issue during this approximation. This is due to a trade-off: increasing the number of breakpoints reduces the error but also increases the computational complexity of the embedded model. As this trade-off is unclear in advance, preliminary experiments are often required to determine these values. The objective of this study is to approximate the trade-off between error and breakpoints in piecewise linearization for first-order loss functions. To achieve this goal, we derive an upper bound on the minimum number of breakpoints required to achieve a given absolute error. This upper bound can be easily precomputed once the approximation intervals and error are determined, and serves as a guideline for the trade-off between error and breakpoints. Furthermore, we propose efficient algorithms to obtain a piecewise linear approximation with a number of breakpoints below the derived upper bound.