Quantization-Aware Collaborative Inference for Large Embodied AI Models

TL;DR

This paper proposes a quantization-aware collaborative inference method for large embodied AI models, optimizing resource use while maintaining inference quality under delay and energy constraints.

Contribution

It introduces a novel approximation for quantization distortion, derives bounds on rate-distortion, and formulates a joint optimization for bit-width and computation frequency.

Findings

The distortion approximation accurately predicts inference quality loss.

The derived bounds effectively guide resource allocation.

Joint design improves latency and energy efficiency in edge AI systems.

Abstract

Large artificial intelligence models (LAIMs) are increasingly regarded as a core intelligence engine for embodied AI applications. However, the massive parameter scale and computational demands of LAIMs pose significant challenges for resource-limited embodied agents. To address this issue, we investigate quantization-aware collaborative inference (co-inference) for embodied AI systems. First, we develop a tractable approximation for quantization-induced inference distortion. Based on this approximation, we derive lower and upper bounds on the quantization rate-inference distortion function, characterizing its dependence on LAIM statistics, including the quantization bit-width. Next, we formulate a joint quantization bit-width and computation frequency design problem under delay and energy constraints, aiming to minimize the distortion upper bound while ensuring tightness through the corresponding lower bound. Extensive evaluations validate the proposed distortion approximation, the derived rate-distortion bounds, and the effectiveness of the proposed joint design. Particularly, simulations and real-world testbed experiments demonstrate the effectiveness of the proposed joint design in balancing inference quality, latency, and energy consumption in edge embodied AI systems.