Iterative Supervised Learning for Regression with Constraints

TL;DR

This paper introduces an iterative supervised learning method for regression that enforces arbitrary constraints, guarantees convergence, and improves model quality and stability through a novel proof and numerical experiments.

Contribution

It provides the first analytical convergence proof for constrained regression using an iterative approach with alternating steps.

Findings

Convergence of the iterative method is guaranteed under mild assumptions.

Numerical results show improved regression accuracy and constraint satisfaction.

The method enhances training stability compared to existing algorithms.

Abstract

Regression in supervised learning often requires the enforcement of constraints to ensure that the trained models are consistent with the underlying structures of the input and output data. This paper presents an iterative procedure to perform regression under arbitrary constraints. It is achieved by alternating between a learning step and a constraint enforcement step, to which an affine extension function is incorporated. We show this leads to a contraction mapping under mild assumptions, from which the convergence is guaranteed analytically. The presented proof of convergence in regression with constraints is the unique contribution of this paper. Furthermore, numerical experiments illustrate improvements in the trained model in terms of the quality of regression, the satisfaction of constraints, and also the stability in training, when compared to other existing algorithms.