Online and Stable Learning of Analysis Operators

TL;DR

This paper introduces four novel iterative algorithms for learning analysis operators, demonstrating improved recovery rates and faster convergence on synthetic and image data, with applications in denoising.

Contribution

It presents four new algorithms for analysis operator learning based on a unified optimization principle, including a step-size free method and a computationally efficient variant.

Findings

All algorithms decrease or preserve the target function each step.

They outperform Analysis SimCO in recovery rates and convergence speed.

They perform comparably or better than state-of-the-art in denoising tasks.

Abstract

In this paper four iterative algorithms for learning analysis operators are presented. They are built upon the same optimisation principle underlying both Analysis K-SVD and Analysis SimCO. The Forward and Sequential Analysis Operator Learning (FAOL and SAOL) algorithms are based on projected gradient descent with optimally chosen step size. The Implicit AOL (IAOL) algorithm is inspired by the implicit Euler scheme for solving ordinary differential equations and does not require to choose a step size. The fourth algorithm, Singular Value AOL (SVAOL), uses a similar strategy as Analysis K-SVD while avoiding its high computational cost. All algorithms are proven to decrease or preserve the target function in each step and a characterisation of their stationary points is provided. Further they are tested on synthetic and image data, compared to Analysis SimCO and found to give better recovery rates and faster decay of the objective function respectively. In a final denoising experiment the presented algorithms are again shown to perform similar to or better than the state-of-the-art algorithm ASimCO.