A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraints

TL;DR

This paper introduces a simple, computationally efficient method for embedding hard convex constraints into neural network outputs by adding a specialized layer that guarantees feasibility, applicable to various constraint types.

Contribution

The paper presents a novel neural network layer that enforces hard convex constraints on outputs, extending to joint and dynamic constraints with low computational complexity.

Findings

Effective in solving constrained optimization problems

Applicable to linear, quadratic, and dynamic constraints

Demonstrated through classification and optimization tasks

Abstract

A new computationally simple method of imposing hard convex constraints on the neural network output values is proposed. The key idea behind the method is to map a vector of hidden parameters of the network to a point that is guaranteed to be inside the feasible set defined by a set of constraints. The mapping is implemented by the additional neural network layer with constraints for output. The proposed method is simply extended to the case when constraints are imposed not only on the output vectors, but also on joint constraints depending on inputs. The projection approach to imposing constraints on outputs can simply be implemented in the framework of the proposed method. It is shown how to incorporate different types of constraints into the proposed method, including linear and quadratic constraints, equality constraints, and dynamic constraints, constraints in the form of boundaries. An important feature of the method is its computational simplicity. Complexities of the forward pass of the proposed neural network layer by linear and quadratic constraints are O(n*m) and O(n^2*m), respectively, where n is the number of variables, m is the number of constraints. Numerical experiments illustrate the method by solving optimization and classification problems. The code implementing the method is publicly available.