Computing a Sparse Projection into a Box

TL;DR

This paper presents an efficient algorithm to project vectors onto the intersection of a sparsity constraint and a box, useful for trust-region methods in nonsmooth optimization, with a practical Julia implementation.

Contribution

It introduces a novel $O(n \,\log n)$ algorithm for projecting onto a nonconvex set combining sparsity and box constraints, with applications in optimization.

Findings

Projection computed in $O(n \log n)$ time.

Implementation demonstrated in trust-region optimization methods.

Applicable to nonsmooth regularized optimization problems.

Abstract

We describe a procedure to compute a projection of $w \in \mathbb{R}^n$ into the intersection of the so-called \emph{zero-norm} ball $k \mathbb{B}_0$ of radius $k$, i.e., the set of $k$-sparse vectors, with a box centered at a point of $k \mathbb{B}_0$. The need for such projection arises in the context of certain trust-region methods for nonsmooth regularized optimization. Although the set into which we wish to project is nonconvex, we show that a solution may be found in $O(n \log(n))$ operations. We describe our Julia implementation and illustrate our procedure in the context of two trust-region methods for nonsmooth regularized optimization.