MobileNVC: Real-time 1080p Neural Video Compression on a Mobile Device
Ties van Rozendaal, Tushar Singhal, Hoang Le, Guillaume Sautiere, Amir, Said, Krishna Buska, Anjuman Raha, Dimitris Kalatzis, Hitarth Mehta, Frank, Mayer, Liang Zhang, Markus Nagel, Auke Wiggers

TL;DR
This paper introduces MobileNVC, a neural video codec capable of real-time 1080p decoding on mobile devices by optimizing computational efficiency and leveraging hardware acceleration, outperforming previous codecs significantly.
Contribution
The paper presents the first neural video codec that decodes 1080p video in real time on mobile devices, using block-based motion compensation and model quantization for efficiency.
Findings
Achieves real-time 1080p decoding on mobile devices.
Outperforms previous codecs with up to 48% BD-rate savings.
Reduces MAC count on receiver side by 10x.
Abstract
Neural video codecs have recently become competitive with standard codecs such as HEVC in the low-delay setting. However, most neural codecs are large floating-point networks that use pixel-dense warping operations for temporal modeling, making them too computationally expensive for deployment on mobile devices. Recent work has demonstrated that running a neural decoder in real time on mobile is feasible, but shows this only for 720p RGB video. This work presents the first neural video codec that decodes 1080p YUV420 video in real time on a mobile device. Our codec relies on two major contributions. First, we design an efficient codec that uses a block-based motion compensation algorithm available on the warping core of the mobile accelerator, and we show how to quantize this model to integer precision. Second, we implement a fast decoder pipeline that concurrently runs neural network…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
MobileNVC: Real-Time 1080p Neural Video Compression on a Mobile Device· youtube
Taxonomy
TopicsAdvanced Vision and Imaging · Video Coding and Compression Technologies · Advanced Data Compression Techniques
