Exploiting the Structure via Sketched Gradient Algorithms

TL;DR

This paper introduces a convergence analysis for sketched gradient algorithms, demonstrating their efficiency in large-scale constrained least-squares problems by exploiting low-dimensional geometric structures.

Contribution

It provides the first convergence analysis showing linear rates for the Gradient Projection Classical Sketch method, highlighting its efficiency in large-scale constrained regression.

Findings

GPCS converges linearly to a solution vicinity.

Trade-offs exist between computational cost and sketch size.

The method effectively exploits low-dimensional structures.

Abstract

Sketched gradient algorithms have been recently introduced for efficiently solving the large-scale constrained Least-squares regressions. In this paper we provide novel convergence analysis for the basic method {\it Gradient Projection Classical Sketch} (GPCS) to reveal the fast linear convergence rate of GPCS towards a vicinity of the solution thanks to the intrinsic low-dimensional geometric structure of the solution prompted by constraint set. Similar to our analysis we observe computational and sketch size trade-offs in numerical experiments. Hence we justify that the combination of gradient methods and the sketching technique is a way of designing efficient algorithms which can actively exploit the low-dimensional structure to accelerate computation in large scale data regression and signal processing applications.