STAMP: Scalable Task And Model-agnostic Collaborative Perception

TL;DR

STAMP introduces a scalable, model-agnostic collaborative perception framework for autonomous driving that efficiently integrates heterogeneous agents, improving perception accuracy while reducing computational costs and maintaining security.

Contribution

It presents the first task- and model-agnostic pipeline for multi-agent perception, enabling efficient feature sharing among diverse models with minimal overhead.

Findings

Achieves comparable or better accuracy than state-of-the-art methods.

Reduces computational costs significantly.

Supports heterogeneous agent integration effectively.

Abstract

Perception is crucial for autonomous driving, but single-agent perception is often constrained by sensors' physical limitations, leading to degraded performance under severe occlusion, adverse weather conditions, and when detecting distant objects. Multi-agent collaborative perception offers a solution, yet challenges arise when integrating heterogeneous agents with varying model architectures. To address these challenges, we propose STAMP, a scalable task- and model-agnostic, collaborative perception pipeline for heterogeneous agents. STAMP utilizes lightweight adapter-reverter pairs to transform Bird's Eye View (BEV) features between agent-specific and shared protocol domains, enabling efficient feature sharing and fusion. This approach minimizes computational overhead, enhances scalability, and preserves model security. Experiments on simulated and real-world datasets demonstrate STAMP's comparable or superior accuracy to state-of-the-art models with significantly reduced computational costs. As a first-of-its-kind task- and model-agnostic framework, STAMP aims to advance research in scalable and secure mobility systems towards Level 5 autonomy. Our project page is at https://xiangbogaobarry.github.io/STAMP and the code is available at https://github.com/taco-group/STAMP.