Dual-to-kernel learning with ideals

TL;DR

This paper introduces a unified theory connecting kernel learning and symbolic algebra, leveraging their duality to develop algorithms for manifold and feature learning, demonstrating improved accuracy on various datasets.

Contribution

It presents a novel duality-based framework that combines algebraic structure-awareness with kernel efficiency, along with two new algorithms, IPCA and AVICA.

Findings

Algorithms achieve high accuracy on synthetic data

Effective manifold and feature learning demonstrated on real data

Unification enhances interpretability and efficiency

Abstract

In this paper, we propose a theory which unifies kernel learning and symbolic algebraic methods. We show that both worlds are inherently dual to each other, and we use this duality to combine the structure-awareness of algebraic methods with the efficiency and generality of kernels. The main idea lies in relating polynomial rings to feature space, and ideals to manifolds, then exploiting this generative-discriminative duality on kernel matrices. We illustrate this by proposing two algorithms, IPCA and AVICA, for simultaneous manifold and feature learning, and test their accuracy on synthetic and real world data.