On Quantizing Neural Representation for Variable-Rate Video Coding

TL;DR

NeuroQuant is a novel post-training quantization method for variable-rate neural video coding that adjusts pre-trained weights' quantization parameters to achieve efficient compression and fast encoding without retraining.

Contribution

This paper introduces the first variable-rate INR-VC method with a theoretical framework for sensitivity-based quantization and novel calibration strategies, improving compression and speed.

Findings

Outperforms existing quantization techniques in variable bitwidths

Enables encoding acceleration up to eight times

Achieves minimal loss with INT2 quantization

Abstract

This work introduces NeuroQuant, a novel post-training quantization (PTQ) approach tailored to non-generalized Implicit Neural Representations for variable-rate Video Coding (INR-VC). Unlike existing methods that require extensive weight retraining for each target bitrate, we hypothesize that variable-rate coding can be achieved by adjusting quantization parameters (QPs) of pre-trained weights. Our study reveals that traditional quantization methods, which assume inter-layer independence, are ineffective for non-generalized INR-VC models due to significant dependencies across layers. To address this, we redefine variable-rate INR-VC as a mixed-precision quantization problem and establish a theoretical framework for sensitivity criteria aimed at simplified, fine-grained rate control. Additionally, we propose network-wise calibration and channel-wise quantization strategies to minimize quantization-induced errors, arriving at a unified formula for representation-oriented PTQ calibration. Our experimental evaluations demonstrate that NeuroQuant significantly outperforms existing techniques in varying bitwidth quantization and compression efficiency, accelerating encoding by up to eight times and enabling quantization down to INT2 with minimal reconstruction loss. This work introduces variable-rate INR-VC for the first time and lays a theoretical foundation for future research in rate-distortion optimization, advancing the field of video coding technology. The materials will be available at https://github.com/Eric-qi/NeuroQuant.