Federated Smoothing Proximal Gradient for Quantile Regression with Non-Convex Penalties

TL;DR

This paper presents a federated quantile regression algorithm that combines smoothing and proximal gradient techniques to effectively handle nonconvex penalties and non-smooth loss functions in distributed IoT data analysis.

Contribution

It introduces the federated smoothing proximal gradient (FSPG) algorithm, a novel method that improves estimation accuracy and convergence in federated quantile regression with nonconvex penalties.

Findings

Enhanced estimation precision demonstrated in simulations

Reliable convergence achieved across distributed devices

Effective handling of nonconvex penalties like MCP and SCAD

Abstract

Distributed sensors in the internet-of-things (IoT) generate vast amounts of sparse data. Analyzing this high-dimensional data and identifying relevant predictors pose substantial challenges, especially when data is preferred to remain on the device where it was collected for reasons such as data integrity, communication bandwidth, and privacy. This paper introduces a federated quantile regression algorithm to address these challenges. Quantile regression provides a more comprehensive view of the relationship between variables than mean regression models. However, traditional approaches face difficulties when dealing with nonconvex sparse penalties and the inherent non-smoothness of the loss function. For this purpose, we propose a federated smoothing proximal gradient (FSPG) algorithm that integrates a smoothing mechanism with the proximal gradient framework, thereby enhancing both precision and computational speed. This integration adeptly handles optimization over a network of devices, each holding local data samples, making it particularly effective in federated learning scenarios. The FSPG algorithm ensures steady progress and reliable convergence in each iteration by maintaining or reducing the value of the objective function. By leveraging nonconvex penalties, such as the minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD), the proposed method can identify and preserve key predictors within sparse models. Comprehensive simulations validate the robust theoretical foundations of the proposed algorithm and demonstrate improved estimation precision and reliable convergence.