Neural Video Compression with Domain Transfer

TL;DR

This paper introduces DCVC-DT, a neural video compression framework with online domain transfer and dynamic rate-distortion adjustment, improving performance and generalization across different data domains.

Contribution

It proposes a lightweight online domain transfer mechanism and a frame-level RD adjustment scheme, enhancing neural video compression without retraining the encoder or decoder.

Findings

Achieves up to 6.21% bitrate savings over baseline.

Improves generalization to unseen data.

Reduces error propagation in neural video coding.

Abstract

Content-adaptive compression has always been a key direction in neural video coding (NVC), aiming to mitigate the domain gap between training and testing data. Such gaps often arise from distributional discrepancies between training and inference data, which may cause noticeable performance degradation when the testing content differs from the training distribution. To tackle this challenge, we propose DCVC-DT, a domain transfer enhanced neural video compression framework. Specifically, we design a lightweight online domain transfer (DT) mechanism that dynamically adapts the encoded latent representation during inference, effectively bridging the domain gap without modifying the encoder or decoder parameters. In addition, we develop a frame-level dynamic RD (Rate and Distortion) adjustment scheme that actively regulates the ratio of R and D in the loss function based on quality fluctuation, thereby improving rate-distortion performance. Extensive experiments demonstrate that DCVC-DT achieves up to 6.21% bitrate savings over the baseline DCVC-DC, while significantly enhancing generalization to unseen testing data and alleviating error propagation. Our code is available at https://github.com/SunnyMass/DCVC-DT.