# Sparse multiresolution representations with adaptive kernels

## TL;DR

This paper introduces a novel sparse multiresolution approach with adaptive kernels for RKHS-based methods, enabling flexible, efficient, and locally adaptive function estimation that overcomes prior limitations in complexity, smoothness heterogeneity, and kernel selection.

## Contribution

It proposes a new integral representation and sparse functional programming framework for adaptive, multiresolution kernel methods with theoretical guarantees and practical efficiency.

## Key findings

- Exact and efficient solutions via duality for non-convex sparse problems
- Improved handling of heterogeneous smoothness in functions
- Reduced computational complexity in kernel-based estimation

## Abstract

Reproducing kernel Hilbert spaces (RKHSs) are key elements of many non-parametric tools successfully used in signal processing, statistics, and machine learning. In this work, we aim to address three issues of the classical RKHS based techniques. First, they require the RKHS to be known a priori, which is unrealistic in many applications. Furthermore, the choice of RKHS affects the shape and smoothness of the solution, thus impacting its performance. Second, RKHSs are ill-equipped to deal with heterogeneous degrees of smoothness, i.e., with functions that are smooth in some parts of their domain but vary rapidly in others. Finally, the computational complexity of evaluating the solution of these methods grows with the number of data points, rendering these techniques infeasible for many applications. Though kernel learning, local kernel adaptation, and sparsity have been used to address these issues, many of these approaches are computationally intensive or forgo optimality guarantees. We tackle these problems by leveraging a novel integral representation of functions in RKHSs that allows for arbitrary centers and different kernels at each center. To address the complexity issues, we then write the function estimation problem as a sparse functional program that explicitly minimizes the support of the representation leading to low complexity solutions. Despite their non-convexity and infinite dimensionality, we show these problems can be solved exactly and efficiently by leveraging duality, and we illustrate this new approach in simulated and real data.