Slimmable Video Codec

TL;DR

This paper introduces SlimVC, a slimmable neural video codec that dynamically adjusts model capacity to balance rate-distortion performance with computational and memory efficiency, making neural video compression more practical.

Contribution

The paper proposes a novel slimmable video codec integrating a slimmable temporal entropy model within a slimmable autoencoder, enabling flexible resource management without sacrificing RD performance.

Findings

SlimVC effectively balances rate, memory, and computation.

SlimVC maintains competitive RD performance across different model sizes.

SlimVC demonstrates practical advantages in resource-constrained environments.

Abstract

Neural video compression has emerged as a novel paradigm combining trainable multilayer neural networks and machine learning, achieving competitive rate-distortion (RD) performances, but still remaining impractical due to heavy neural architectures, with large memory and computational demands. In addition, models are usually optimized for a single RD tradeoff. Recent slimmable image codecs can dynamically adjust their model capacity to gracefully reduce the memory and computation requirements, without harming RD performance. In this paper we propose a slimmable video codec (SlimVC), by integrating a slimmable temporal entropy model in a slimmable autoencoder. Despite a significantly more complex architecture, we show that slimming remains a powerful mechanism to control rate, memory footprint, computational cost and latency, all being important requirements for practical video compression.