ADMM for Structured Fractional Minimization

TL;DR

This paper introduces FADMM, a novel ADMM-based algorithm for structured fractional minimization problems with nonconvex and nonsmooth components, demonstrating convergence and superior performance in machine learning applications.

Contribution

The paper develops the first ADMM-based method tailored for structured fractional minimization, with convergence guarantees and variants using Dinkelbach's and quadratic transform methods.

Findings

FADMM converges to ε-approximate critical points within O(1/ε^3) iterations.

The method outperforms existing subgradient and smoothing proximal gradient methods.

Experimental results show effectiveness in applications like sparse Fisher discriminant analysis and robust sparse recovery.

Abstract

This paper considers a class of structured fractional minimization problems. The numerator consists of a differentiable function, a simple nonconvex nonsmooth function, a concave nonsmooth function, and a convex nonsmooth function composed with a linear operator. The denominator is a continuous function that is either weakly convex or has a weakly convex square root. These problems are prevalent in various important applications in machine learning and data science. Existing methods, primarily based on subgradient methods and smoothing proximal gradient methods, often suffer from slow convergence and numerical stability issues. In this paper, we introduce {\sf FADMM}, the first Alternating Direction Method of Multipliers tailored for this class of problems. {\sf FADMM} decouples the original problem into linearized proximal subproblems, featuring two variants: one using Dinkelbach's parametric method ({\sf FADMM-D}) and the other using the quadratic transform method ({\sf FADMM-Q}). By introducing a novel Lyapunov function, we establish that {\sf FADMM} converges to $\epsilon$-approximate critical points of the problem within an oracle complexity of $\mathcal{O}(1/\epsilon^{3})$. Extensive experiments on synthetic and real-world datasets, including sparse Fisher discriminant analysis, robust Sharpe ratio minimization, and robust sparse recovery, demonstrate the effectiveness of our approach. Keywords: Fractional Minimization, Nonconvex Optimization, Proximal Linearized ADMM, Nonsmooth Optimization, Convergence Analysis