Reliable Remote Inference from Unreliable Components: Joint Communication and Computation Limits

TL;DR

This paper investigates the limits of reliable remote inference when both communication channels and receiver components are unreliable, establishing fundamental bounds and conditions for achieving reliable inference.

Contribution

It introduces a novel information-theoretic framework modeling unreliable receiver components and derives new converse bounds and achievability results under different closure assumptions.

Findings

The supply-demand converse bounds the task-relevant information by the minimum of communication and compute budgets.

Committed interfaces create additional cuts, affecting the information flow and reliability.

Achievability results are established for certain closure models, but some regimes remain open.

Abstract

Classical information theory typically assumes reliable receiver-side processing. We study remote inference when communication is noisy and the receiver itself is built from unreliable components under a finite redundancy budget. Under a committed/no-bypass receiver closure, task-relevant information can affect the final estimate only by passing through a budgeted collection of vulnerable primitives unless an explicit protected bypass is modeled. Modeling each vulnerable primitive as a memoryless noisy channel yields a baseline supply--demand converse: the task-relevant information needed to attain a target distortion cannot exceed the smaller of the total information supplied by the communication channel and the total information supplied by the vulnerable compute budget. Our main converse shows that committed intermediate interfaces create additional first-order serial cuts and receiver-internal computation-graph cuts, captured in general by a receiver-internal compute min-cut converse. In particular, the twofold loss in the symmetric two-stage hard-separation special case is not inherent to unreliable receiver computation but induced by hard-separation under the committed/no-bypass closure. This extra first-order tax is therefore closure-dependent rather than universal. On the converse side, if downstream modules retain soft visibility to the raw channel output, the converse reduces to the single-bottleneck supply, up to any explicitly reserved soft-path budget. Under a separate stronger protected-support closure with reliable decoder and control support, we establish achievability results for task-direct and serial hard-separation constructions. For the fully noisy-logic regime, we obtain only a conservative depth-dependent converse, and matched achievability remains open.