Rate-Accuracy Bounds in Visual Coding for Machines

TL;DR

This paper derives theoretical rate-accuracy bounds for visual coding tailored for machine analysis, revealing significant gaps between current methods and optimal bounds, indicating substantial room for improvement.

Contribution

It introduces a theoretical framework for rate-accuracy bounds in visual coding for machines and compares these bounds with existing methods, highlighting their inefficiency.

Findings

Current methods are 10 to 1000 times less efficient than theoretical bounds.

There is significant potential for improving visual coding strategies for machine analysis.

Theoretical bounds provide a benchmark for future development in the field.

Abstract

Increasingly, visual signals such as images, videos and point clouds are being captured solely for the purpose of automated analysis by computer vision models. Applications include traffic monitoring, robotics, autonomous driving, smart home, and many others. This trend has led to the need to develop compression strategies for these signals for the purpose of analysis rather than reconstruction, an area often referred to as "coding for machines." By drawing parallels with lossy coding of a discrete memoryless source, in this paper we derive rate-accuracy bounds on several popular problems in visual coding for machines, and compare these with state-of-the-art results from the literature. The comparison shows that the current results are at least an order of magnitude -- and in some cases two or three orders of magnitude -- away from the theoretical bounds in terms of the bitrate needed to achieve a certain level of accuracy. This, in turn, means that there is much room for improvement in the current methods for visual coding for machines.