On the Characteristics of the Conjugate Function Enabling Effective Dual Decomposition Methods

TL;DR

This paper introduces the FGOR characteristic of conjugate functions, enabling more efficient dual decomposition methods with improved convergence and reduced communication, applicable to convex and nonconvex problems.

Contribution

The paper identifies the FGOR property of conjugate functions, leveraging it to enhance dual subgradient methods and develop a simple stepsize rule that improves efficiency and communication in distributed optimization.

Findings

FGOR accelerates convergence of dual methods.

FGOR reduces communication overhead in distributed settings.

Numerical results show significant performance improvements.

Abstract

We investigate a novel characteristic of the conjugate function associated to a generic convex optimization problem, which can subsequently be leveraged for efficient dual decomposition methods. In particular, under mild assumptions, we show that there is a specific region in the domain of the conjugate function such that for any point in the region, there is always a ray originating from that point along which the gradients of the conjugate remain constant. We refer to this characteristic as a fixed gradient over rays (FGOR). We further show that this characteristic is inherited by the corresponding dual function. Then we provide a thorough exposition of the application of the FGOR characteristic to dual subgradient methods. More importantly, we leverage FGOR to devise a simple stepsize rule that can be prepended with state-of-the-art stepsize methods enabling them to be more efficient. Furthermore, we investigate how the FGOR characteristic is used when solving the global consensus problem, a prevalent formulation in diverse application domains. We show that FGOR can be exploited not only to expedite the convergence of the dual decomposition methods but also to reduce the communication overhead. FGOR is extended to nonconvex formulations, and its advantages in stochastic optimization are demonstrated. Numerical experiments using quadratic objectives and a regularized least squares regression with real datasets are conducted. The results show that FGOR can significantly improve the performance of existing stepsize methods and outperform the state-of-the-art splitting methods on average in terms of both convergence behavior and communication efficiency.