Sample-wise Constrained Learning via a Sequential Penalty Approach with Applications in Image Processing

TL;DR

This paper introduces a sequential penalty method for constrained learning that effectively handles sample-specific constraints, with proven convergence and practical success in image processing applications.

Contribution

It proposes a novel sequential penalty algorithm for sample-wise constraints with convergence guarantees suitable for deep learning, demonstrated through image processing experiments.

Findings

Convergence guarantees under reasonable deep learning assumptions.

Effective handling of sample-specific constraints in practice.

Successful application to image processing tasks.

Abstract

In many learning tasks, certain requirements on the processing of individual data samples should arguably be formalized as strict constraints in the underlying optimization problem, rather than by means of arbitrary penalties. We show that, in these scenarios, learning can be carried out exploiting a sequential penalty method that allows to properly deal with constraints. The proposed algorithm is shown to possess convergence guarantees under assumptions that are reasonable in deep learning scenarios. Moreover, the results of experiments on image processing tasks show that the method is indeed viable to be used in practice.