Penalty Dual Decomposition Method For Nonsmooth Nonconvex Optimization

TL;DR

This paper introduces the penalty dual decomposition (PDD) algorithm for solving complex nonconvex nonsmooth optimization problems common in signal processing, machine learning, and wireless communications, with proven convergence and practical applications.

Contribution

The paper proposes a novel double-loop PDD algorithm that effectively addresses nonconvex nonsmooth problems and establishes its convergence to KKT solutions.

Findings

PDD converges to KKT solutions in nonconvex nonsmooth optimization.

PDD demonstrates effectiveness in signal processing and wireless communication applications.

The algorithm outperforms existing methods in specific problem settings.

Abstract

Many contemporary signal processing, machine learning and wireless communication applications can be formulated as nonconvex nonsmooth optimization problems. Often there is a lack of efficient algorithms for these problems, especially when the optimization variables are nonlinearly coupled in some nonconvex constraints. In this work, we propose an algorithm named penalty dual decomposition (PDD) for these difficult problems and discuss its various applications. The PDD is a double-loop iterative algorithm. Its inner iterations is used to inexactly solve a nonconvex nonsmooth augmented Lagrangian problem via block-coordinate-descenttype methods, while its outer iteration updates the dual variables and/or a penalty parameter. In Part I of this work, we describe the PDD algorithm and rigorously establish its convergence to KKT solutions. In Part II we evaluate the performance of PDD by customizing it to three applications arising from signal processing and wireless communications.