Douglas-Rachford splitting and ADMM for nonconvex optimization: Accelerated and Newton-type linesearch algorithms

TL;DR

This paper introduces accelerated linesearch algorithms using quasi-Newton directions to improve the robustness and convergence of Douglas-Rachford splitting and ADMM for nonconvex optimization, especially under ill conditioning.

Contribution

It proposes two new linesearch algorithms with quasi-Newton directions that enhance DRS and ADMM, maintaining convergence and achieving superlinear rates under certain conditions.

Findings

L-BFGS improves convergence robustness.

Algorithms maintain convergence properties for nonconvex problems.

Superlinear convergence achieved with Broyden directions.

Abstract

Although the performance of popular optimization algorithms such as Douglas-Rachford splitting (DRS) and the ADMM is satisfactory in small and well-scaled problems, ill conditioning and problem size pose a severe obstacle to their reliable employment. Expanding on recent convergence results for DRS and ADMM applied to nonconvex problems, we propose two linesearch algorithms to enhance and robustify these methods by means of quasi-Newton directions. The proposed algorithms are suited for nonconvex problems, require the same black-box oracle of DRS and ADMM, and maintain their (subsequential) convergence properties. Numerical evidence shows that the employment of L-BFGS in the proposed framework greatly improves convergence of DRS and ADMM, making them robust to ill conditioning. Under regularity and nondegeneracy assumptions at the limit point, superlinear convergence is shown when quasi-Newton Broyden directions are adopted.