Column Partition based Distributed Algorithms for Coupled Convex Sparse Optimization: Dual and Exact Regularization Approaches

TL;DR

This paper introduces column partition based distributed algorithms for large-scale convex sparse optimization problems, leveraging dual and exact regularization methods to enable efficient solutions when decision variables vastly outnumber measurements.

Contribution

It develops novel distributed schemes for coupled convex sparse problems using dual and regularization approaches, addressing limitations of existing methods for large-scale, densely coupled problems.

Findings

Proposed schemes effectively solve large-scale sparse optimization problems.

Convergence of the algorithms is theoretically established.

Numerical results demonstrate the schemes' efficiency and accuracy.

Abstract

This paper develops column partition based distributed schemes for a class of large-scale convex sparse optimization problems, e.g., basis pursuit (BP), LASSO, basis pursuit denosing (BPDN), and their extensions, e.g., fused LASSO. We are particularly interested in the cases where the number of (scalar) decision variables is much larger than the number of (scalar) measurements, and each agent has limited memory or computing capacity such that it only knows a small number of columns of a measurement matrix. These problems in consideration are densely coupled and cannot be formulated as separable convex programs using column partition. To overcome this difficulty, we consider their dual problems which are separable or locally coupled. Once a dual solution is attained, it is shown that a primal solution can be found from the dual of corresponding regularized BP-like problems under suitable exact regularization conditions. A wide range of existing distributed schemes can be exploited to solve the obtained dual problems. This yields two-stage column partition based distributed schemes for LASSO-like and BPDN-like problems; the overall convergence of these schemes is established using sensitivity analysis techniques. Numerical results illustrate the effectiveness of the proposed schemes.