# Adaptive Iterative Hessian Sketch via A-Optimal Subsampling

## TL;DR

This paper introduces an improved deterministic iterative Hessian sketch method using A-optimal subsampling, enhancing efficiency and accuracy for large-scale least squares problems through novel initialization, preconditioning, and adaptive step sizing.

## Contribution

The paper proposes a deterministic A-optimal subsampling approach to enhance the iterative Hessian sketch for large-scale least squares, including new initialization, preconditioning, and step size methods.

## Key findings

- Outperforms existing accelerated IHS methods in experiments.
- Provides a deterministic alternative to randomized sketching.
- Demonstrates improved computational efficiency and accuracy.

## Abstract

Iterative Hessian sketch (IHS) is an effective sketching method for modeling large-scale data. It was originally proposed by Pilanci and Wainwright (2016; JMLR) based on randomized sketching matrices. However, it is computationally intensive due to the iterative sketch process. In this paper, we analyze the IHS algorithm under the unconstrained least squares problem setting, then propose a deterministic approach for improving IHS via A-optimal subsampling. Our contributions are three-fold: (1) a good initial estimator based on the A-optimal design is suggested; (2) a novel ridged preconditioner is developed for repeated sketching; and (3) an exact line search method is proposed for determining the optimal step length adaptively. Extensive experimental results demonstrate that our proposed A-optimal IHS algorithm outperforms the existing accelerated IHS methods.