Parameterized Convex Universal Approximators for Decision-Making Problems

TL;DR

This paper introduces parameterized convex neural networks, PMA and PLSE, that serve as universal approximators for decision-making problems, extending existing convex models with continuous parameter functions and demonstrating superior performance in high-dimensional scenarios.

Contribution

The paper proposes novel parameterized convex neural networks, PMA and PLSE, with proven universal approximation capabilities and practical guidelines for their integration with deep learning.

Findings

PLSE outperforms existing approximators in minimizer and optimal value errors.

The proposed models are scalable and efficient for high-dimensional problems.

Universal approximation theorem established for PMA and PLSE.

Abstract

Parameterized max-affine (PMA) and parameterized log-sum-exp (PLSE) networks are proposed for general decision-making problems. The proposed approximators generalize existing convex approximators, namely, max-affine (MA) and log-sum-exp (LSE) networks, by considering function arguments of condition and decision variables and replacing the network parameters of MA and LSE networks with continuous functions with respect to the condition variable. The universal approximation theorem of PMA and PLSE is proven, which implies that PMA and PLSE are shape-preserving universal approximators for parameterized convex continuous functions. Practical guidelines for incorporating deep neural networks within PMA and PLSE networks are provided. A numerical simulation is performed to demonstrate the performance of the proposed approximators. The simulation results support that PLSE outperforms other existing approximators in terms of minimizer and optimal value errors with scalable and efficient computation for high-dimensional cases.