New VVC profiles targeting Feature Coding for Machines

TL;DR

This paper explores VVC-based compression of neural network features for machine vision, proposing three profiles that balance efficiency and accuracy improvements.

Contribution

It introduces three lightweight VVC profiles tailored for feature coding, optimizing compression and processing speed for machine vision tasks.

Findings

Fast profile achieves 2.96% BD-Rate gain with 21.8% faster encoding.

Faster profile achieves 1.85% BD-Rate gain with 51.5% speedup.

Fastest profile reduces encoding time by 95.6% with minimal BD-Rate loss.

Abstract

Modern video codecs have been extensively optimized to preserve perceptual quality, leveraging models of the human visual system. However, in split inference systems-where intermediate features from neural network are transmitted instead of pixel data-these assumptions no longer apply. Intermediate features are abstract, sparse, and task-specific, making perceptual fidelity irrelevant. In this paper, we investigate the use of Versatile Video Coding (VVC) for compressing such features under the MPEG-AI Feature Coding for Machines (FCM) standard. We perform a tool-level analysis to understand the impact of individual coding components on compression efficiency and downstream vision task accuracy. Based on these insights, we propose three lightweight essential VVC profiles-Fast, Faster, and Fastest. The Fast profile provides 2.96% BD-Rate gain while reducing encoding time by 21.8%. Faster achieves a 1.85% BD-Rate gain with a 51.5% speedup. Fastest reduces encoding time by 95.6% with only a 1.71% loss in BD-Rate.