Sparse solution of overdetermined linear systems when the columns of $A$ are orthogonal

TL;DR

This paper presents an efficient method for finding exact sparse solutions to overdetermined linear systems with orthogonal columns, improving computational efficiency over brute-force approaches.

Contribution

It introduces a novel approach that efficiently computes exact sparse solutions for systems with orthogonal columns, surpassing traditional exponential-time methods.

Findings

Exact solutions can be obtained with less computational effort

Orthogonal columns enable simplified sparse solution computation

Method outperforms naive exponential search

Abstract

In this paper, we consider the problem of obtaining the best $k$-sparse solution of $Ax=y$ subject to the constraint that the columns of $A$ are orthogonal. The naive approach for obtaining a solution to this problem has exponential complexity and there exist $l_1$ regularization methods such as Lasso to obtain approximate solutions. In this paper, we show that we can obtain an exact solution to the problem, with much less computational effort compared to the brute force search when the columns of $A$ are orthogonal.