QuantVSR: Low-Bit Post-Training Quantization for Real-World Video Super-Resolution

TL;DR

QuantVSR introduces a low-bit quantization approach for real-world video super-resolution, utilizing a spatio-temporal complexity aware mechanism and bias alignment to maintain high performance while reducing resource consumption.

Contribution

The paper presents a novel quantization method with a complexity-aware layer-specific rank allocation and learnable bias alignment for VSR models.

Findings

Achieves comparable performance to full-precision models.

Significantly outperforms recent low-bit quantization methods.

Effective on both synthetic and real-world datasets.

Abstract

Diffusion models have shown superior performance in real-world video super-resolution (VSR). However, the slow processing speeds and heavy resource consumption of diffusion models hinder their practical application and deployment. Quantization offers a potential solution for compressing the VSR model. Nevertheless, quantizing VSR models is challenging due to their temporal characteristics and high fidelity requirements. To address these issues, we propose QuantVSR, a low-bit quantization model for real-world VSR. We propose a spatio-temporal complexity aware (STCA) mechanism, where we first utilize the calibration dataset to measure both spatial and temporal complexities for each layer. Based on these statistics, we allocate layer-specific ranks to the low-rank full-precision (FP) auxiliary branch. Subsequently, we jointly refine the FP and low-bit branches to achieve simultaneous optimization. In addition, we propose a learnable bias alignment (LBA) module to reduce the biased quantization errors. Extensive experiments on synthetic and real-world datasets demonstrate that our method obtains comparable performance with the FP model and significantly outperforms recent leading low-bit quantization methods. Code is available at: https://github.com/bowenchai/QuantVSR.