Effortless Cross-Platform Video Codec: A Codebook-Based Method

TL;DR

This paper introduces a cross-platform video codec that uses codebooks and a cross-attention mechanism, eliminating entropy models and ensuring consistent decoding across different hardware platforms, with competitive compression performance.

Contribution

The proposed framework removes the need for entropy models and optical flow, enabling efficient, cross-platform video compression with consistent decoding and improved performance over traditional codecs.

Findings

Outperforms H.265 (medium) in compression quality.

Eliminates entropy model inconsistencies across platforms.

Achieves high efficiency without optical flow or autoregressive models.

Abstract

Under certain circumstances, advanced neural video codecs can surpass the most complex traditional codecs in their rate-distortion (RD) performance. One of the main reasons for the high performance of existing neural video codecs is the use of the entropy model, which can provide more accurate probability distribution estimations for compressing the latents. This also implies the rigorous requirement that entropy models running on different platforms should use consistent distribution estimations. However, in cross-platform scenarios, entropy models running on different platforms usually yield inconsistent probability distribution estimations due to floating point computation errors that are platform-dependent, which can cause the decoding side to fail in correctly decoding the compressed bitstream sent by the encoding side. In this paper, we propose a cross-platform video compression framework based on codebooks, which avoids autoregressive entropy modeling and achieves video compression by transmitting the index sequence of the codebooks. Moreover, instead of using optical flow for context alignment, we propose to use the conditional cross-attention module to obtain the context between frames. Due to the absence of autoregressive modeling and optical flow alignment, we can design an extremely minimalist framework that can greatly benefit computational efficiency. Importantly, our framework no longer contains any distribution estimation modules for entropy modeling, and thus computations across platforms are not necessarily consistent. Experimental results show that our method can outperform the traditional H.265 (medium) even without any entropy constraints, while achieving the cross-platform property intrinsically.