TSFormer: A Robust Framework for Efficient UHD Image Restoration

TL;DR

TSFormer is a novel UHD image restoration framework that combines trusted learning and sparsification, enabling real-time processing of 4K images with high quality, robustness, and efficiency.

Contribution

Introduces TSFormer, a framework integrating trusted learning with token sparsification, improving generalization and efficiency in UHD image restoration.

Findings

Achieves real-time 4K image restoration at 40fps

Outperforms existing methods in restoration quality

Reduces computational complexity significantly

Abstract

Ultra-high-definition (UHD) image restoration is vital for applications demanding exceptional visual fidelity, yet existing methods often face a trade-off between restoration quality and efficiency, limiting their practical deployment. In this paper, we propose TSFormer, an all-in-one framework that integrates \textbf{T}rusted learning with \textbf{S}parsification to boost both generalization capability and computational efficiency in UHD image restoration. The key is that only a small amount of token movement is allowed within the model. To efficiently filter tokens, we use Min-$p$ with random matrix theory to quantify the uncertainty of tokens, thereby improving the robustness of the model. Our model can run a 4K image in real time (40fps) with 3.38 M parameters. Extensive experiments demonstrate that TSFormer achieves state-of-the-art restoration quality while enhancing generalization and reducing computational demands. In addition, our token filtering method can be applied to other image restoration models to effectively accelerate inference and maintain performance.