Partition-Insensitive Parallel ADMM Algorithm for High-dimensional Linear Models

TL;DR

This paper introduces a partition-insensitive parallel ADMM algorithm based on linearized ADMM for high-dimensional regression, reducing computational complexity and maintaining solution stability regardless of data partitioning.

Contribution

The paper proposes a novel partition-insensitive parallel ADMM framework that reduces auxiliary variables and is applicable to nonconvex high-dimensional regression problems.

Findings

Algorithm reduces the number of variables needed at each iteration.

Solution remains stable regardless of data partitioning.

Numerical experiments confirm effectiveness on synthetic and real data.

Abstract

The parallel alternating direction method of multipliers (ADMM) algorithms have gained popularity in statistics and machine learning due to their efficient handling of large sample data problems. However, the parallel structure of these algorithms, based on the consensus problem, can lead to an excessive number of auxiliary variables when applied to highdimensional data, resulting in large computational burden. In this paper, we propose a partition-insensitive parallel framework based on the linearized ADMM (LADMM) algorithm and apply it to solve nonconvex penalized high-dimensional regression problems. Compared to existing parallel ADMM algorithms, our algorithm does not rely on the consensus problem, resulting in a significant reduction in the number of variables that need to be updated at each iteration. It is worth noting that the solution of our algorithm remains largely unchanged regardless of how the total sample is divided, which is known as partition-insensitivity. Furthermore, under some mild assumptions, we prove the convergence of the iterative sequence generated by our parallel algorithm. Numerical experiments on synthetic and real datasets demonstrate the feasibility and validity of the proposed algorithm. We provide a publicly available R software package to facilitate the implementation of the proposed algorithm.