Simultaneous Best Subset Selection and Dimension Reduction via Primal-Dual Iterations

TL;DR

This paper introduces a new primal-dual algorithm for sparse reduced rank regression that achieves near-optimal estimation rates, supports consistent support and rank recovery, and demonstrates scalability through numerical and real data experiments.

Contribution

The paper proposes a novel primal-dual iterative algorithm for sparse reduced rank regression with theoretical guarantees and practical effectiveness.

Findings

Achieves near-optimal estimation rates.

Ensures support and rank recovery consistency.

Demonstrates scalability in numerical and real data applications.

Abstract

Sparse reduced rank regression is an essential statistical learning method. In the contemporary literature, estimation is typically formulated as a nonconvex optimization that often yields to a local optimum in numerical computation. Yet, their theoretical analysis is always centered on the global optimum, resulting in a discrepancy between the statistical guarantee and the numerical computation. In this research, we offer a new algorithm to address the problem and establish an almost optimal rate for the algorithmic solution. We also demonstrate that the algorithm achieves the estimation with a polynomial number of iterations. In addition, we present a generalized information criterion to simultaneously ensure the consistency of support set recovery and rank estimation. Under the proposed criterion, we show that our algorithm can achieve the oracle reduced rank estimation with a significant probability. The numerical studies and an application in the ovarian cancer genetic data demonstrate the effectiveness and scalability of our approach.