EV-NVC: Efficient Variable bitrate Neural Video Compression

TL;DR

This paper presents EV-NVC, a neural video codec that uses novel modules and training strategies to improve variable bitrate compression, achieving significant BD-rate reduction over traditional methods.

Contribution

Introduction of EV-NVC with piecewise linear sampler and long-short-term feature fusion, along with mixed-precision training strategies for improved rate-distortion performance.

Findings

Reduces BD-rate by 30.56% compared to HM-16.25.

Enhances rate-distortion performance in high bitrate range.

Improves context modeling with LSTFFM.

Abstract

Training neural video codec (NVC) with variable rate is a highly challenging task due to its complex training strategies and model structure. In this paper, we train an efficient variable bitrate neural video codec (EV-NVC) with the piecewise linear sampler (PLS) to improve the rate-distortion performance in high bitrate range, and the long-short-term feature fusion module (LSTFFM) to enhance the context modeling. Besides, we introduce mixed-precision training and discuss the different training strategies for each stage in detail to fully evaluate its effectiveness. Experimental results show that our approach reduces the BD-rate by 30.56% compared to HM-16.25 within low-delay mode.