Exact Dual Geometry of SOC-ICNN Value Functions

TL;DR

This paper explores the geometric properties of SOC-ICNN value functions, enabling explicit, white-box inference by leveraging dual variables, and validates these methods through numerical experiments.

Contribution

It provides a dual viewpoint analysis of SOC-ICNNs, deriving geometric primitives from optimal dual variables for exact, transparent inference.

Findings

Supporting slopes and subdifferentials can be recovered from dual variables.

Local Hessians are explicitly computed from dual solutions.

Numerical experiments confirm the validity of the theoretical formulas.

Abstract

Input Convex Neural Networks (ICNNs) are commonly used in a two-stage manner: one first trains a convex network and then minimizes it over its input in a downstream inference problem. Recent second-order-cone ICNNs (SOC-ICNNs) enrich ReLU-based ICNNs with quadratic and conic modules and admit an exact representation as value functions of second-order cone programs (SOCPs). This value-function structure enables an explicit convex-analytic treatment of SOC-ICNN inference. In this paper, we study the exact first-order and local second-order geometry of SOC-ICNNs from the dual viewpoint. We show that supporting slopes, subdifferentials, directional derivatives, and local Hessians can be recovered directly from optimal dual variables. These results provide the geometric primitives for white-box SOC-ICNN inference, going beyond black-box automatic differentiation. Numerical experiments validate the exact multiplier readout, the local Hessian formula, and the set-valued behavior at structurally degenerate inputs. We also provide a step-by-step tutorial showing how the readout mechanism instantiates a complete white-box inference loop. The code is available at https://anonymous.4open.science/r/SOC-ICNN-Theory-BEFC/.