POLICE: Provably Optimal Linear Constraint Enforcement for Deep Neural Networks

TL;DR

POLICE is a novel method that enforces affine constraints on deep neural networks during training and testing, requiring minimal modifications and ensuring constraint satisfaction without sampling, thus improving the reliability of constrained DNNs.

Contribution

This paper introduces POLICE, the first provably optimal affine constraint enforcement technique for DNNs that is computationally efficient and compatible with standard training procedures.

Findings

Ensures DNNs satisfy affine constraints during training and testing.

Requires minimal changes to the DNN's forward pass.

Does not need sampling and guarantees constraint fulfillment.

Abstract

Deep Neural Networks (DNNs) outshine alternative function approximators in many settings thanks to their modularity in composing any desired differentiable operator. The formed parametrized functional is then tuned to solve a task at hand from simple gradient descent. This modularity comes at the cost of making strict enforcement of constraints on DNNs, e.g. from a priori knowledge of the task, or from desired physical properties, an open challenge. In this paper we propose the first provable affine constraint enforcement method for DNNs that only requires minimal changes into a given DNN's forward-pass, that is computationally friendly, and that leaves the optimization of the DNN's parameter to be unconstrained, i.e. standard gradient-based method can be employed. Our method does not require any sampling and provably ensures that the DNN fulfills the affine constraint on a given input space's region at any point during training, and testing. We coin this method POLICE, standing for Provably Optimal LInear Constraint Enforcement. Github: https://github.com/RandallBalestriero/POLICE