On Structured Filtering-Clustering: Global Error Bound and Optimal First-Order Algorithms

TL;DR

This paper establishes a global error bound for filtering-clustering models, introduces optimal first-order algorithms, and provides convergence analysis and experiments demonstrating their efficiency in solving these models.

Contribution

It identifies a global error bound condition for dual filtering-clustering problems and designs optimal first-order algorithms with proven convergence rates.

Findings

Global error bound condition holds for many filtering-clustering problems.

Proposed algorithms are optimal in deterministic, finite-sum, and online settings.

Experiments confirm the effectiveness of the algorithms on real datasets.

Abstract

The filtering-clustering models, including trend filtering and convex clustering, have become an important source of ideas and modeling tools in machine learning and related fields. The statistical guarantee of optimal solutions in these models has been extensively studied yet the investigations on the computational aspect have remained limited. In particular, practitioners often employ the first-order algorithms in real-world applications and are impressed by their superior performance regardless of ill-conditioned structures of difference operator matrices, thus leaving open the problem of understanding the convergence property of first-order algorithms. This paper settles this open problem and contributes to the broad interplay between statistics and optimization by identifying a \textit{global error bound} condition, which is satisfied by a large class of dual filtering-clustering problems, and designing a class of \textit{generalized dual gradient ascent} algorithm, which is \textit{optimal} first-order algorithms in deterministic, finite-sum and online settings. Our results are new and help explain why the filtering-clustering models can be efficiently solved by first-order algorithms. We also provide the detailed convergence rate analysis for the proposed algorithms in different settings, shedding light on their potential to solve the filtering-clustering models efficiently. We also conduct experiments on real datasets and the numerical results demonstrate the effectiveness of our algorithms.