An Optimization Framework with Flexible Inexact Inner Iterations for Nonconvex and Nonsmooth Programming

TL;DR

This paper introduces a flexible optimization framework called IPAD for nonconvex and nonsmooth programming, providing theoretical convergence guarantees and demonstrating superior practical performance.

Contribution

The paper proposes a novel inexact proximal alternating direction method (IPAD) that can incorporate various algorithms for subproblems with guaranteed convergence for general NNPs.

Findings

IPAD guarantees convergence under simple error conditions.

Numerical experiments show IPAD's superiority and flexibility.

Framework effectively handles real-world and synthetic data.

Abstract

In recent years, numerous vision and learning tasks have been (re)formulated as nonconvex and nonsmooth programmings(NNPs). Although some algorithms have been proposed for particular problems, designing fast and flexible optimization schemes with theoretical guarantee is a challenging task for general NNPs. It has been investigated that performing inexact inner iterations often benefit to special applications case by case, but their convergence behaviors are still unclear. Motivated by these practical experiences, this paper designs a novel algorithmic framework, named inexact proximal alternating direction method (IPAD) for solving general NNPs. We demonstrate that any numerical algorithms can be incorporated into IPAD for solving subproblems and the convergence of the resulting hybrid schemes can be consistently guaranteed by a series of simple error conditions. Beyond the guarantee in theory, numerical experiments on both synthesized and real-world data further demonstrate the superiority and flexibility of our IPAD framework for practical use.