Random Projections for the Nonnegative Least-Squares Problem

TL;DR

This paper introduces a fast random projection algorithm using a randomized Hadamard transform to efficiently approximate solutions to the Nonnegative Least Squares problem, achieving speedups with minimal accuracy loss.

Contribution

It presents a novel random projection method for NNLS that reduces problem size and provides theoretical guarantees of near-optimal solutions with high probability.

Findings

Significant speedups in solving NNLS problems demonstrated experimentally.

High-probability guarantees of solution quality close to the true optimum.

Effective approximation of Euclidean lengths using a small subset of matrix rows.

Abstract

Constrained least-squares regression problems, such as the Nonnegative Least Squares (NNLS) problem, where the variables are restricted to take only nonnegative values, often arise in applications. Motivated by the recent development of the fast Johnson-Lindestrauss transform, we present a fast random projection type approximation algorithm for the NNLS problem. Our algorithm employs a randomized Hadamard transform to construct a much smaller NNLS problem and solves this smaller problem using a standard NNLS solver. We prove that our approach finds a nonnegative solution vector that, with high probability, is close to the optimum nonnegative solution in a relative error approximation sense. We experimentally evaluate our approach on a large collection of term-document data and verify that it does offer considerable speedups without a significant loss in accuracy. Our analysis is based on a novel random projection type result that might be of independent interest. In particular, given a tall and thin matrix $\Phi \in \mathbb{R}^{n \times d}$ ($n \gg d$) and a vector $y \in \mathbb{R}^d$, we prove that the Euclidean length of $\Phi y$ can be estimated very accurately by the Euclidean length of $\tilde{\Phi}y$, where $\tilde{\Phi}$ consists of a small subset of (appropriately rescaled) rows of $\Phi$.