Soft-Radial Projection for Constrained End-to-End Learning

TL;DR

This paper introduces Soft-Radial Projection, a differentiable layer that enforces constraints in deep learning models, overcoming gradient saturation issues of traditional projection methods, and improving optimization and solution quality.

Contribution

The paper proposes a novel radial reparameterization layer that maintains full-rank Jacobians and strict feasibility, enhancing constrained end-to-end learning.

Findings

Improved convergence behavior compared to existing methods.

Maintains universal approximation property.

Achieves better solution quality in experiments.

Abstract

Integrating hard constraints into deep learning is essential for safety-critical systems. Yet existing constructive layers that project predictions onto constraint boundaries face a fundamental bottleneck: gradient saturation. By collapsing exterior points onto lower-dimensional surfaces, standard orthogonal projections induce rank-deficient Jacobians, which nullify gradients orthogonal to active constraints and hinder optimization. We introduce Soft-Radial Projection, a differentiable reparameterization layer that circumvents this issue through a radial mapping from Euclidean space into the interior of the feasible set. This construction guarantees strict feasibility while preserving a full-rank Jacobian almost everywhere, thereby preventing the optimization stalls typical of boundary-based methods. We theoretically prove that the architecture retains the universal approximation property and empirically show improved convergence behavior and solution quality over state-of-the-art optimization- and projection-based baselines.