Emerging Standards for Machine-to-Machine Video Coding

TL;DR

This paper reviews emerging standards for machine-to-machine video coding, focusing on task-aware and feature coding methods that improve efficiency, privacy, and performance in visual data transmission between machines.

Contribution

It introduces the design and evaluation of FCM and VCM standards, demonstrating their effectiveness in reducing bitrate while maintaining task accuracy and analyzing codec performance impacts.

Findings

FCM maintains accuracy close to edge inference with lower bitrate.

HEVC and VVC codecs perform similarly in machine tasks, with minimal BD-Rate difference.

Existing hardware codecs like HEVC support machine-to-machine communication without performance loss.

Abstract

Machines are increasingly becoming the primary consumers of visual data, yet most deployments of machine-to-machine systems still rely on remote inference where pixel-based video is streamed using codecs optimized for human perception. Consequently, this paradigm is bandwidth intensive, scales poorly, and exposes raw images to third parties. Recent efforts in the Moving Picture Experts Group (MPEG) redesigned the pipeline for machine-to-machine communication: Video Coding for Machines (VCM) is designed to apply task-aware coding tools in the pixel domain, and Feature Coding for Machines (FCM) is designed to compress intermediate neural features to reduce bitrate, preserve privacy, and support compute offload. Experiments show that FCM is capable of maintaining accuracy close to edge inference while significantly reducing bitrate. Additional analysis of H.26X codecs used as inner codecs in FCM reveals that H.265/High Efficiency Video Coding (HEVC) and H.266/Versatile Video Coding (VVC) achieve almost identical machine task performance, with an average BD-Rate increase of 1.39% when VVC is replaced with HEVC. In contrast, H.264/Advanced Video Coding (AVC) yields an average BD-Rate increase of 32.28% compared to VVC. However, for the tracking task, the impact of codec choice is minimal, with HEVC outperforming VVC and achieving BD Rate of -1.81% and 8.79% for AVC, indicating that existing hardware for already deployed codecs can support machine-to-machine communication without degrading performance.