Reconstructing Sparse Signals via Greedy Monte-Carlo Search

TL;DR

This paper introduces a greedy Monte-Carlo (GMC) algorithm for reconstructing sparse signals in high-dimensional linear regression, demonstrating its effectiveness in noiseless and noisy scenarios through numerical experiments.

Contribution

The paper presents a novel GMC algorithm that explicitly selects variables and accepts updates based on energy decrease, outperforming some existing methods in sparse signal reconstruction.

Findings

GMC achieves perfect reconstruction in noiseless, undersampled cases.

GMC outperforms $\,ell_1$ relaxation in certain scenarios.

GMC is practical for noisy data, as shown by experiments.

Abstract

We propose a Monte-Carlo-based method for reconstructing sparse signals in the formulation of sparse linear regression in a high-dimensional setting. The basic idea of this algorithm is to explicitly select variables or covariates to represent a given data vector or responses and accept randomly generated updates of that selection if and only if the energy or cost function decreases. This algorithm is called the greedy Monte-Carlo (GMC) search algorithm. Its performance is examined via numerical experiments, which suggests that in the noiseless case, GMC can achieve perfect reconstruction in undersampling situations of a reasonable level: it can outperform the $\ell_1$ relaxation but does not reach the algorithmic limit of MC-based methods theoretically clarified by an earlier analysis. The necessary computational time is also examined and compared with that of an algorithm using simulated annealing. Additionally, experiments on the noisy case are conducted on synthetic datasets and on a real-world dataset, supporting the practicality of GMC.