Robust low-rank estimation with multiple binary responses using pairwise AUC loss

TL;DR

This paper introduces a low-rank, pairwise AUC loss-based method for multi-response binary classification that improves ranking performance and shared structure exploitation, especially in high-dimensional, imbalanced data.

Contribution

It proposes a novel unified framework that directly optimizes AUC for multiple binary responses using low-rank constraints and scalable algorithms, outperforming likelihood-based methods.

Findings

Method achieves linear convergence under regularity conditions.

Outperforms likelihood-based approaches in simulations.

Robust to label switching and data contamination.

Abstract

Multiple binary responses arise in many modern data-analytic problems. Although fitting separate logistic regressions for each response is computationally attractive, it ignores shared structure and can be statistically inefficient, especially in high-dimensional and class-imbalanced regimes. Low-rank models offer a natural way to encode latent dependence across tasks, but existing methods for binary data are largely likelihood-based and focus on pointwise classification rather than ranking performance. In this work, we propose a unified framework for learning with multiple binary responses that directly targets discrimination by minimizing a surrogate loss for the area under the ROC curve (AUC). The method aggregates pairwise AUC surrogate losses across responses while imposing a low-rank constraint on the coefficient matrix to exploit shared structure. We develop a scalable projected gradient descent algorithm based on truncated singular value decomposition. Exploiting the fact that the pairwise loss depends only on differences of linear predictors, we simplify computation and analysis. We establish non-asymptotic convergence guarantees, showing that under suitable regularity conditions, leading to linear convergence up to the minimax-optimal statistical precision. Extensive simulation studies demonstrate that the proposed method is robust in challenging settings such as label switching and data contamination and consistently outperforms likelihood-based approaches.