Shorting Dynamics and Structured Kernel Regularization

TL;DR

This paper introduces a dynamic operator framework for structured kernel regularization that enhances invariance and convergence properties in data analysis, unifying several concepts in kernel methods.

Contribution

It develops a nonlinear operator dynamic for structured regularization, connecting residual decompositions with invariant kernel construction in a unified framework.

Findings

Converges to the classical shorted operator.

Provides a canonical form of kernel ridge regression.

Enables principled nuisance invariance enforcement.

Abstract

This paper develops a nonlinear operator dynamic that progressively removes the influence of a prescribed feature subspace while retaining maximal structure elsewhere. The induced sequence of positive operators is monotone, admits an exact residual decomposition, and converges to the classical shorted operator. Transporting this dynamic to reproducing kernel Hilbert spaces yields a corresponding family of kernels that converges to the largest kernel dominated by the original one and annihilating the given subspace. In the finite-sample setting, the associated Gram operators inherit a structured residual decomposition that leads to a canonical form of kernel ridge regression and a principled way to enforce nuisance invariance. This gives a unified operator-analytic approach to invariant kernel construction and structured regularization in data analysis.