Rate-Distortion Theory in Coding for Machines and its Application

TL;DR

This paper extends rate-distortion theory to optimize image and video coding for machine analysis, leading to improved performance in various computer vision tasks.

Contribution

It develops a theoretical framework for machine-specific rate-distortion and applies it to enhance learnable image coding methods.

Findings

Achieved state-of-the-art rate-distortion performance in classification.

Improved coding efficiency for instance segmentation.

Enhanced object detection accuracy with new coding methods.

Abstract

Recent years have seen a tremendous growth in both the capability and popularity of automatic machine analysis of images and video. As a result, a growing need for efficient compression methods optimized for machine vision, rather than human vision, has emerged. To meet this growing demand, several methods have been developed for image and video coding for machines. Unfortunately, while there is a substantial body of knowledge regarding rate-distortion theory for human vision, the same cannot be said of machine analysis. In this paper, we extend the current rate-distortion theory for machines, providing insight into important design considerations of machine-vision codecs. We then utilize this newfound understanding to improve several methods for learnable image coding for machines. Our proposed methods achieve state-of-the-art rate-distortion performance on several computer vision tasks such as classification, instance segmentation, and object detection.