Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression

TL;DR

This paper introduces Group-aware Parameter-efficient Updating (GPU), a novel method for content-adaptive neural video compression that minimizes error accumulation and reduces update costs through group-aware parameter updates and lightweight adapters.

Contribution

The paper proposes a new group-aware, parameter-efficient updating method for neural video compression that improves adaptability and reduces update complexity compared to existing approaches.

Findings

Outperforms existing methods on four video benchmarks

Demonstrates effective adaptation on a medical image benchmark

Reduces update time and computational cost significantly

Abstract

Content-adaptive compression is crucial for enhancing the adaptability of the pre-trained neural codec for various contents. Although these methods have been very practical in neural image compression (NIC), their application in neural video compression (NVC) is still limited due to two main aspects: 1), video compression relies heavily on temporal redundancy, therefore updating just one or a few frames can lead to significant errors accumulating over time; 2), NVC frameworks are generally more complex, with many large components that are not easy to update quickly during encoding. To address the previously mentioned challenges, we have developed a content-adaptive NVC technique called Group-aware Parameter-Efficient Updating (GPU). Initially, to minimize error accumulation, we adopt a group-aware approach for updating encoder parameters. This involves adopting a patch-based Group of Pictures (GoP) training strategy to segment a video into patch-based GoPs, which will be updated to facilitate a globally optimized domain-transferable solution. Subsequently, we introduce a parameter-efficient delta-tuning strategy, which is achieved by integrating several light-weight adapters into each coding component of the encoding process by both serial and parallel configuration. Such architecture-agnostic modules stimulate the components with large parameters, thereby reducing both the update cost and the encoding time. We incorporate our GPU into the latest NVC framework and conduct comprehensive experiments, whose results showcase outstanding video compression efficiency across four video benchmarks and adaptability of one medical image benchmark.