One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception

TL;DR

UniTrans is a universal model enabling zero-shot, any-to-any feature modality translation for collaborative perception, overcoming heterogeneity barriers without retraining across different modalities.

Contribution

Proposes UniTrans, a pre-trained, zero-shot, universal translation model for arbitrary feature modalities in collaborative perception, eliminating the need for retraining.

Findings

Outperforms state-of-the-art methods on OPV2V-H and DAIR-V2X datasets.

Enables efficient zero-shot translation across diverse modalities.

Demonstrates robustness in both simulated and real-world settings.

Abstract

By sharing intermediate features, collaborative perception extends each agent's sensing beyond standalone limits, but real-world feature modality heterogeneity remains a key barrier to effective fusion. Most existing methods, including direct adaption and protocol-based transformation, typically rely on training adapters for newly emerging feature modalities and often require additional retraining or fine-tuning. Such repeated training is costly and is often infeasible across manufacturers due to model and data privacy constraints, limiting real-world scalability. To address this issue, we propose UniTrans, a universal any-to-any feature modality translation model that instantiates translators on the fly for arbitrary modalities. UniTrans pretrains a bank of translator expert parameters and learns their combination coefficients as a function of source-to-target modality mapping. The mapping is measured in a modality-intrinsic latent space, where an intrinsic encoder extracts modality-specific yet scene-invariant codes from single-frame intermediate features, enabling UniTrans to instantiate translators in a zero-shot manner. Experiments on OPV2V-H and DAIR-V2X demonstrate that UniTrans consistently outperforms state-of-the-art methods in both simulated and real-world settings, enabling efficient any-to-any translation through a universal model. The code is available at https://github.com/CheeryLeeyy/UniTrans.