Implementation of an Optimal First-Order Method for Strongly Convex Total Variation Regularization

TL;DR

This paper introduces a practical implementation of Nesterov's optimal first-order method for large-scale total variation regularization, with mechanisms to estimate key parameters and improve performance on ill-conditioned problems.

Contribution

It proposes a novel approach to estimate strong convexity and Lipschitz constants during iterations, enabling effective application to non-strongly convex functions and large-scale problems.

Findings

Significantly outperforms existing first-order methods on ill-conditioned problems.

Demonstrates effectiveness in 3D tomography reconstruction.

Provides theoretical and empirical analysis of iteration complexity.

Abstract

We present a practical implementation of an optimal first-order method, due to Nesterov, for large-scale total variation regularization in tomographic reconstruction, image deblurring, etc. The algorithm applies to $\mu$-strongly convex objective functions with $L$-Lipschitz continuous gradient. In the framework of Nesterov both $\mu$ and $L$ are assumed known -- an assumption that is seldom satisfied in practice. We propose to incorporate mechanisms to estimate locally sufficient $\mu$ and $L$ during the iterations. The mechanisms also allow for the application to non-strongly convex functions. We discuss the iteration complexity of several first-order methods, including the proposed algorithm, and we use a 3D tomography problem to compare the performance of these methods. The results show that for ill-conditioned problems solved to high accuracy, the proposed method significantly outperforms state-of-the-art first-order methods, as also suggested by theoretical results.