Learned ISTA with Error-based Thresholding for Adaptive Sparse Coding

TL;DR

This paper introduces an error-based thresholding mechanism for learned ISTA that improves adaptivity to data variations and accelerates convergence, supported by theoretical analysis and extensive experiments.

Contribution

It proposes an error-based thresholding method for LISTA that enhances adaptivity and convergence speed, with rigorous theoretical and experimental validation.

Findings

Enhanced adaptivity to data variations

Faster convergence of LISTA models

Validated effectiveness through extensive experiments

Abstract

Drawing on theoretical insights, we advocate an error-based thresholding (EBT) mechanism for learned ISTA (LISTA), which utilizes a function of the layer-wise reconstruction error to suggest a specific threshold for each observation in the shrinkage function of each layer. We show that the proposed EBT mechanism well disentangles the learnable parameters in the shrinkage functions from the reconstruction errors, endowing the obtained models with improved adaptivity to possible data variations. With rigorous analyses, we further show that the proposed EBT also leads to a faster convergence on the basis of LISTA or its variants, in addition to its higher adaptivity. Extensive experimental results confirm our theoretical analyses and verify the effectiveness of our methods.