A Self-scaled Approximate $\ell_0$ Regularization Robust Model for Outlier Detection

TL;DR

This paper introduces a novel robust regression model called SARM that employs self-scaled approximate l0 regularization, improving robustness and computational efficiency in outlier detection, especially for large-scale and adversarial data scenarios.

Contribution

The paper proposes a new self-scaled approximate l0 regularization model and an efficient alternating minimization algorithm with convergence guarantees, advancing robust regression methods.

Findings

SARM outperforms existing methods in robustness and efficiency.

TSSARM enhances robustness when design matrix singular values are spread.

Real-world load forecasting shows improved resistance to adversarial attacks.

Abstract

Robust regression models in the presence of outliers have significant practical relevance in areas such as signal processing, financial econometrics, and energy management. Many existing robust regression methods, either grounded in statistical theory or sparse signal recovery, typically rely on the explicit or implicit assumption of outlier sparsity to filter anomalies and recover the underlying signal or data. However, these methods often suffer from limited robustness or high computational complexity, rendering them inefficient for large-scale problems. In this work, we propose a novel robust regression model based on a Self-scaled Approximate l0 Regularization Model (SARM) scheme. By introducing a self-scaling mechanism into the regularization term, the proposed model mitigates the negative impact of uneven or excessively large outlier magnitudes on robustness. We also develop an alternating minimization algorithm grounded in Proximal Operators and Block Coordinate Descent. We rigorously prove the algorithm convergence. Empirical comparisons with several state-of-the-art robust regression methods demonstrate that SARM not only achieves superior robustness but also significantly improves computational efficiency. Motivated by both the theoretical error bound and empirical observations, we further design a Two-Stage SARM (TSSARM) framework, which better utilizes sample information when the singular values of the design matrix are widely spread, thereby enhancing robustness under certain conditions. Finally, we validate our approach on a real-world load forecasting task. The experimental results show that our method substantially enhances the robustness of load forecasting against adversarial data attacks, which is increasingly critical in the era of heightened data security concerns.