Normalization of ReLU Dual for Cut Generation in Stochastic Mixed-Integer Programs

TL;DR

This paper introduces a normalization technique for the ReLU dual in stochastic mixed-integer programs, producing stronger cuts and improving computational efficiency.

Contribution

It proposes a dual normalization method that enhances cut strength and convergence in stochastic mixed-integer programming, outperforming existing regularization approaches.

Findings

Normalized cuts are tight and Pareto-optimal.

Normalization recovers any cut from regularization methods.

The approach reduces solution times on complex instances.

Abstract

We study the Rectified Linear Unit (ReLU) dual, an existing dual formulation for stochastic programs that reformulates non-anticipativity constraints using ReLU functions to generate tight, non-convex, and mixed-integer representable cuts. While this dual reformulation guarantees convergence with mixed-integer state variables, it admits multiple optimal solutions that can yield weak cuts. To address this issue, we propose normalizing the dual in the extended space to identify solutions that yield stronger cuts. We prove that the resulting normalized cuts are tight and Pareto-optimal in the original state space. We further compare normalization with existing regularization-based approaches for handling dual degeneracy and explain why normalization offers key advantages. In particular, we show that normalization can recover any cut obtained via regularization, whereas the converse does not hold. Computational experiments demonstrate that the proposed approach outperforms existing methods by consistently yielding stronger cuts and reducing solution times on harder instances.