SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions

TL;DR

The paper introduces SOC-ICNN, a novel neural network architecture that extends input convex neural networks from polyhedral to conic geometry, enhancing their representational capacity with smooth curvature.

Contribution

It generalizes ICNNs from linear programming to second-order cone programming, increasing expressiveness while maintaining computational efficiency.

Findings

SOC-ICNN significantly improves function approximation accuracy.

The architecture delivers competitive decision-making performance.

It expands the representational space without increasing asymptotic complexity.

Abstract

Classical ReLU-based Input Convex Neural Networks (ICNNs) are equivalent to the optimal value functions of Linear Programming (LP). This intrinsic structural equivalence restricts their representational capacity to piecewise-linear polyhedral functions. To overcome this representational bottleneck, we propose the SOC-ICNN, an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP). By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native smooth curvature into the representation while preserving a rigorous optimization-theoretic interpretation. We formally prove that SOC-ICNNs strictly expand the representational space of ReLU-ICNNs without increasing the asymptotic order of forward-pass complexity. Extensive experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality. The code is available at https://anonymous.4open.science/r/SOC-ICNN-4B18/.