Greedy coordinate descent method on non-negative quadratic programming

TL;DR

This paper introduces a greedy coordinate descent method for non-negative quadratic programming that converges faster overall than cyclic or randomized methods, with applications to constrained problems and matrix factorization.

Contribution

It demonstrates that greedy coordinate descent can achieve faster convergence in large-scale non-negative quadratic programming compared to traditional methods.

Findings

Greedy CD has faster overall convergence than cyclic and randomized methods.

The method is effective for linearly constrained NQP and non-negative matrix factorization.

Numerical results show promising performance on synthetic and image data.

Abstract

The coordinate descent (CD) method has recently become popular for solving very large-scale problems, partly due to its simple update, low memory requirement, and fast convergence. In this paper, we explore the greedy CD on solving non-negative quadratic programming (NQP). The greedy CD generally has much more expensive per-update complexity than its cyclic and randomized counterparts. However, on the NQP, these three CDs have almost the same per-update cost, while the greedy CD can have significantly faster overall convergence speed. We also apply the proposed greedy CD as a subroutine to solve linearly constrained NQP and the non-negative matrix factorization. Promising numerical results on both problems are observed on instances with synthetic data and also image data.