Learning with Algebraic Invariances, and the Invariant Kernel Trick

TL;DR

This paper introduces a framework for embedding algebraic invariances into kernel methods, enabling more effective data analysis by leveraging prior structural knowledge through a double kernel trick.

Contribution

It presents a novel approach to incorporate algebraic invariances into kernels using a double kernel trick, applicable to various data analysis tasks.

Findings

Effective incorporation of sign symmetries and phase invariance in kernels

Improved spectral clustering and ICA performance with invariance-aware kernels

Theoretical foundation for algebraic invariance integration in kernel methods

Abstract

When solving data analysis problems it is important to integrate prior knowledge and/or structural invariances. This paper contributes by a novel framework for incorporating algebraic invariance structure into kernels. In particular, we show that algebraic properties such as sign symmetries in data, phase independence, scaling etc. can be included easily by essentially performing the kernel trick twice. We demonstrate the usefulness of our theory in simulations on selected applications such as sign-invariant spectral clustering and underdetermined ICA.