A Similarity Measure Between Functions with Applications to Statistical Learning and Optimization

TL;DR

This paper introduces a new similarity measure between functions that captures how their sub-optimality gaps relate, unifying existing notions and demonstrating applications in empirical risk minimization and online optimization.

Contribution

The paper proposes a novel, unified similarity measure for functions, with practical rules and applications in optimization tasks.

Findings

The measure effectively captures functional similarity in various contexts.

It simplifies analysis in empirical risk minimization.

It aids in non-stationary online optimization scenarios.

Abstract

In this note, we present a novel measure of similarity between two functions. It quantifies how the sub-optimality gaps of two functions convert to each other, and unifies several existing notions of functional similarity. We show that it has convenient operation rules, and illustrate its use in empirical risk minimization and non-stationary online optimization.