Smooth Sparse Coding via Marginal Regression for Learning Sparse Representations

TL;DR

This paper introduces a new sparse coding framework combining kernel smoothing and marginal regression, enhancing scalability, speed, and accuracy in learning sparse representations from data.

Contribution

The paper presents a novel smooth sparse coding method that integrates kernel smoothing and marginal regression, providing theoretical bounds and practical improvements over existing techniques.

Findings

Improved speed and scalability in sparse coding.

Enhanced accuracy demonstrated on real datasets.

Theoretical generalization bounds for the proposed method.

Abstract

We propose and analyze a novel framework for learning sparse representations, based on two statistical techniques: kernel smoothing and marginal regression. The proposed approach provides a flexible framework for incorporating feature similarity or temporal information present in data sets, via non-parametric kernel smoothing. We provide generalization bounds for dictionary learning using smooth sparse coding and show how the sample complexity depends on the L1 norm of kernel function used. Furthermore, we propose using marginal regression for obtaining sparse codes, which significantly improves the speed and allows one to scale to large dictionary sizes easily. We demonstrate the advantages of the proposed approach, both in terms of accuracy and speed by extensive experimentation on several real data sets. In addition, we demonstrate how the proposed approach could be used for improving semi-supervised sparse coding.