Equivalence of Learning Algorithms

TL;DR

This paper introduces the concept of equivalence between machine learning algorithms, defining weak and strong forms, to facilitate transfer of learning properties, exemplified through kernel methods like KRR and M-RLSR.

Contribution

It proposes a formal framework for algorithmic equivalence and demonstrates its application using kernel regression algorithms.

Findings

Defined weak and strong equivalence notions

Analyzed relation between KRR and M-RLSR

Showed transferability of learning properties

Abstract

The purpose of this paper is to introduce a concept of equivalence between machine learning algorithms. We define two notions of algorithmic equivalence, namely, weak and strong equivalence. These notions are of paramount importance for identifying when learning prop erties from one learning algorithm can be transferred to another. Using regularized kernel machines as a case study, we illustrate the importance of the introduced equivalence concept by analyzing the relation between kernel ridge regression (KRR) and m-power regularized least squares regression (M-RLSR) algorithms.