Ultra-lightweight Neural Video Representation Compression

TL;DR

This paper introduces NVRC-Lite, a lightweight neural video compression method that enhances performance and speed by integrating multi-scale feature grids and an octree-based entropy coding model, outperforming existing codecs.

Contribution

The paper proposes NVRC-Lite, combining multi-scale feature grids and an octree-based entropy model to improve lightweight neural video compression performance and efficiency.

Findings

Outperforms C3 with up to 21% BD-rate savings in PSNR.

Achieves 8.4x faster encoding and 2.5x faster decoding.

Demonstrates superior performance at low complexity levels.

Abstract

Recent works have demonstrated the viability of utilizing over-fitted implicit neural representations (INRs) as alternatives to autoencoder-based models for neural video compression. Among these INR-based video codecs, Neural Video Representation Compression (NVRC) was the first to adopt a fully end-to-end compression framework that compresses INRs, achieving state-of-the-art performance. Moreover, some recently proposed lightweight INRs have shown comparable performance to their baseline codecs with computational complexity lower than 10kMACs/pixel. In this work, we extend NVRC toward lightweight representations, and propose NVRC-Lite, which incorporates two key changes. Firstly, we integrated multi-scale feature grids into our lightweight neural representation, and the use of higher resolution grids significantly improves the performance of INRs at low complexity. Secondly, we address the issue that existing INRs typically leverage autoregressive models for entropy coding: these are effective but impractical due to their slow coding speed. In this work, we propose an octree-based context model for entropy coding high-dimensional feature grids, which accelerates the entropy coding module of the model. Our experimental results demonstrate that NVRC-Lite outperforms C3, one of the best lightweight INR-based video codecs, with up to 21.03% and 23.06% BD-rate savings when measured in PSNR and MS-SSIM, respectively, while achieving 8.4x encoding and 2.5x decoding speedup. The implementation of NVRC-Lite will be made available.