One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perception

TL;DR

PolyInter is a novel polymorphic feature interpreter that enables flexible, low-loss interpretation of heterogeneous agents' features in collaborative perception, improving accuracy and adaptability with minimal additional training.

Contribution

It introduces PolyInter, a single adaptable interpreter that handles diverse agents by overriding learnable prompts, enhancing extensibility and reducing training overhead in heterogeneous scenarios.

Findings

Improves perception accuracy by up to 11.1% over SOTA interpreters.

Requires training only 1.4% of parameters for new agents.

Demonstrates effective interpretation in heterogeneous autonomous driving scenarios.

Abstract

Collaborative perception in autonomous driving significantly enhances the perception capabilities of individual agents. Immutable heterogeneity, where agents have different and fixed perception networks, presents a major challenge due to the semantic gap in exchanged intermediate features without modifying the perception networks. Most existing methods bridge the semantic gap through interpreters. However, they either require training a new interpreter for each new agent type, limiting extensibility, or rely on a two-stage interpretation via an intermediate standardized semantic space, causing cumulative semantic loss. To achieve both extensibility in immutable heterogeneous scenarios and low-loss feature interpretation, we propose PolyInter, a polymorphic feature interpreter. It provides an extension point where new agents integrate by overriding only their specific prompts, which are learnable parameters that guide interpretation, while reusing PolyInter's remaining parameters. By leveraging polymorphism, our design enables a single interpreter to accommodate diverse agents and interpret their features into the ego agent's semantic space. Experiments on the OPV2V dataset demonstrate that PolyInter improves collaborative perception precision by up to 11.1% compared to SOTA interpreters, while comparable results can be achieved by training only 1.4% of PolyInter's parameters when adapting to new agents. Code is available at https://github.com/yuchen-xia/PolyInter.