Model-Aware Rate-Distortion Limits for Task-Oriented Source Coding

TL;DR

This paper investigates the fundamental limits of task-oriented source coding for machine inference, introducing new bounds that consider task model suboptimality and architecture constraints, and highlights the gap between current schemes and these limits.

Contribution

It presents task model-aware rate-distortion bounds for TOSC, addressing limitations of previous bounds and analyzing the impact of model suboptimality and complexity.

Findings

Current learned TOSC schemes are far from theoretical limits.

Transmitter-side complexity is a key bottleneck in practical TOSC.

Existing bounds often rely on unrealistic assumptions about task identifiability.

Abstract

Task-Oriented Source Coding (TOSC) has emerged as a paradigm for efficient visual data communication in machine-centric inference systems, where bitrate, latency, and task performance must be jointly optimized under resource constraints. While recent works have proposed rate-distortion bounds for coding for machines, these results often rely on strong assumptions on task identifiability and neglect the impact of deployed task models. In this work, we revisit the fundamental limits of single-TOSC through the lens of indirect rate-distortion theory. We highlight the conditions under which existing rate-distortion bounds are achievable and show their limitations in realistic settings. We then introduce task model-aware rate-distortion bounds that account for task model suboptimality and architectural constraints. Experiments on standard classification benchmarks confirm that current learned TOSC schemes operate far from these limits, highlighting transmitter-side complexity as a key bottleneck.