ShiftLUT: Spatial Shift Enhanced Look-Up Tables for Efficient Image Restoration

TL;DR

ShiftLUT introduces a spatial shift enhanced LUT framework that significantly enlarges the receptive field and improves image restoration performance while maintaining efficiency suitable for edge devices.

Contribution

It proposes a novel framework with learnable spatial shifts, an asymmetric dual-branch architecture, and LUT compression, achieving the largest receptive field and better performance with low overhead.

Findings

3.8× larger receptive field than TinyLUT

Over 0.21 dB PSNR improvement on benchmarks

Maintains small storage and fast inference

Abstract

Look-Up Table based methods have emerged as a promising direction for efficient image restoration tasks. Recent LUT-based methods focus on improving their performance by expanding the receptive field. However, they inevitably introduce extra computational and storage overhead, which hinders their deployment in edge devices. To address this issue, we propose ShiftLUT, a novel framework that attains the largest receptive field among all LUT-based methods while maintaining high efficiency. Our key insight lies in three complementary components. First, Learnable Spatial Shift module (LSS) is introduced to expand the receptive field by applying learnable, channel-wise spatial offsets on feature maps. Second, we propose an asymmetric dual-branch architecture that allocates more computation to the information-dense branch, substantially reducing inference latency without compromising restoration quality. Finally, we incorporate a feature-level LUT compression strategy called Error-bounded Adaptive Sampling (EAS) to minimize the storage overhead. Compared to the previous state-of-the-art method TinyLUT, ShiftLUT achieves a 3.8$\times$ larger receptive field and improves an average PSNR by over 0.21 dB across multiple standard benchmarks, while maintaining a small storage size and inference time. The code is available at: https://github.com/Sailor-t/ShiftLUT .