Decoder-Free Distillation for Quantized Image Restoration

TL;DR

This paper introduces QDR, a novel framework for quantized image restoration that overcomes key challenges in low-level vision tasks, enabling efficient edge deployment with minimal performance loss.

Contribution

It proposes Decoder-Free Distillation and Learnable Magnitude Reweighting to address capacity mismatch, error amplification, and optimization conflicts in quantized IR models.

Findings

Achieves 96.5% of FP32 performance in Int8 models.

Runs at 442 FPS on NVIDIA Jetson Orin.

Improves object detection mAP by 16.3%.

Abstract

Quantization-Aware Training (QAT), combined with Knowledge Distillation (KD), holds immense promise for compressing models for edge deployment. However, joint optimization for precision-sensitive image restoration (IR) to recover visual quality from degraded images remains largely underexplored. Directly adapting QAT-KD to low-level vision reveals three critical bottlenecks: teacher-student capacity mismatch, spatial error amplification during decoder distillation, and an optimization "tug-of-war" between reconstruction and distillation losses caused by quantization noise. To tackle these, we introduce Quantization-aware Distilled Restoration (QDR), a framework for edge-deployed IR. QDR eliminates capacity mismatch via FP32 self-distillation and prevents error amplification through Decoder-Free Distillation (DFD), which corrects quantization errors strictly at the network bottleneck. To stabilize the optimization tug-of-war, we propose a Learnable Magnitude Reweighting (LMR) that dynamically balances competing gradients. Finally, we design an Edge-Friendly Model (EFM) featuring a lightweight Learnable Degradation Gating (LDG) to dynamically modulate spatial degradation localization. Extensive experiments across four IR tasks demonstrate that our Int8 model recovers 96.5% of FP32 performance, achieves 442 frames per second (FPS) on an NVIDIA Jetson Orin, and boosts downstream object detection by 16.3 mAP