Flexible Multi-layer Sparse Approximations of Matrices and Applications

TL;DR

This paper presents a novel algorithm for approximately factorizing high-dimensional matrices into sparse factors, significantly reducing computational costs in signal processing and machine learning applications.

Contribution

It introduces a non-convex optimization-based method for multi-layer sparse matrix approximations, with detailed analysis and practical demonstrations.

Findings

Effective in reducing computational complexity

Applicable to dictionary learning for image denoising

Useful for approximating large matrices in inverse problems

Abstract

The computational cost of many signal processing and machine learning techniques is often dominated by the cost of applying certain linear operators to high-dimensional vectors. This paper introduces an algorithm aimed at reducing the complexity of applying linear operators in high dimension by approximately factorizing the corresponding matrix into few sparse factors. The approach relies on recent advances in non-convex optimization. It is first explained and analyzed in details and then demonstrated experimentally on various problems including dictionary learning for image denoising, and the approximation of large matrices arising in inverse problems.