Optimal Subsampling for High-dimensional Ridge Regression

TL;DR

This paper develops an optimal subsampling method for high-dimensional ridge regression, improving computational efficiency and accuracy through a two-step iterative algorithm based on A-optimal design.

Contribution

It introduces a nearly optimal subsampling scheme and a two-step iterative algorithm for efficient high-dimensional ridge regression.

Findings

The proposed method achieves higher accuracy than existing subsampling techniques.

Numerical experiments confirm the effectiveness and efficiency of the approach.

Theoretical analysis supports the optimality of the subsampling probabilities.

Abstract

We investigate the feature compression of high-dimensional ridge regression using the optimal subsampling technique. Specifically, based on the basic framework of random sampling algorithm on feature for ridge regression and the A-optimal design criterion, we first obtain a set of optimal subsampling probabilities. Considering that the obtained probabilities are uneconomical, we then propose the nearly optimal ones. With these probabilities, a two step iterative algorithm is established which has lower computational cost and higher accuracy. We provide theoretical analysis and numerical experiments to support the proposed methods. Numerical results demonstrate the decent performance of our methods.