Sampling approach to sparse approximation problem: determining degrees of freedom by simulated annealing

TL;DR

This paper introduces a Monte Carlo sampling method using simulated annealing to efficiently solve high-dimensional sparse approximation problems, optimizing the number of selected columns for better generalization.

Contribution

It presents a novel SA-based approach for determining degrees of freedom in sparse approximation, combining it with cross-validation for improved performance.

Findings

SA can find nearly optimal solutions in synthetic tests

Combining SA with CV improves generalization

Method successfully applied to real-world supernova data

Abstract

The approximation of a high-dimensional vector by a small combination of column vectors selected from a fixed matrix has been actively debated in several different disciplines. In this paper, a sampling approach based on the Monte Carlo method is presented as an efficient solver for such problems. Especially, the use of simulated annealing (SA), a metaheuristic optimization algorithm, for determining degrees of freedom (the number of used columns) by cross validation is focused on and tested. Test on a synthetic model indicates that our SA-based approach can find a nearly optimal solution for the approximation problem and, when combined with the CV framework, it can optimize the generalization ability. Its utility is also confirmed by application to a real-world supernova data set.