Tailored max-out networks for learning convex PWQ functions

TL;DR

This paper demonstrates that convex PWQ functions, common in control, can be exactly represented by a simple max-out neural network with one hidden layer and two neurons, facilitating efficient learning.

Contribution

It introduces a neural network topology that can exactly represent convex PWQ functions using a single hidden layer with two neurons, advancing learning efficiency.

Findings

Convex PWQ functions can be exactly modeled by max-out neural networks.

A single hidden layer with two neurons suffices for representation.

This approach simplifies learning of control-related functions.

Abstract

Convex piecewise quadratic (PWQ) functions frequently appear in control and elsewhere. For instance, it is well-known that the optimal value function (OVF) as well as Q-functions for linear MPC are convex PWQ functions. Now, in learning-based control, these functions are often represented with the help of artificial neural networks (NN). In this context, a recurring question is how to choose the topology of the NN in terms of depth, width, and activations in order to enable efficient learning. An elegant answer to that question could be a topology that, in principle, allows to exactly describe the function to be learned. Such solutions are already available for related problems. In fact, suitable topologies are known for piecewise affine (PWA) functions that can, for example, reflect the optimal control law in linear MPC. Following this direction, we show in this paper that convex PWQ functions can be exactly described by max-out-NN with only one hidden layer and two neurons.