Statistic-Augmented, Decoupled MoE Routing and Aggregating in Autonomous Driving

TL;DR

This paper introduces MoE-RAM, a statistic-augmented, decoupled routing and aggregating mechanism for Mixture of Experts models, significantly improving autonomous driving semantic segmentation by enhancing expert selection and output fusion.

Contribution

The paper proposes a novel MoE-RAM mechanism that decouples routing and aggregating, using statistical retrieval and distance-based reweighting to improve expert selection and fusion in autonomous driving tasks.

Findings

MoE-RAM outperforms baseline MoE models in autonomous driving segmentation.

Statistical retrieval improves expert routing accuracy.

Adaptive reweighting enhances prediction quality.

Abstract

Autonomous driving (AD) scenarios are inherently complex and diverse, posing significant challenges for a single deep learning model to effectively cover all possible conditions, such as varying weather, traffic densities, and road types. Large Model (LM)-Driven Mixture of Experts (MoE) paradigm offers a promising solution, where LM serves as the backbone to extract latent features while MoE serves as the downstream head to dynamically select and aggregate specialized experts to adapt to different scenarios. However, routing and aggregating in MoE face intrinsic challenges, including imprecise expert selection due to flawed routing strategy and inefficient expert aggregation leading to suboptimal prediction. To address these issues, we propose a statistic-augmented, decoupled MoE }outing and Aggregating Mechanism (MoE-RAM) driven by LM. Specifically, on the one hand, MoE-RAM enhances expert routing by incorporating statistical retrieval mechanism to match LM-extracted latent features with cached prototypical features of the most relevant experts; on the other hand, MoE-RAM adaptively reweights experts' outputs in fusion by measuring statistical distances of experts' instant features against LM-extracted latent features. Benefiting from the synergy of the statistic-augmented MoE's routing and aggregating, MoE-RAM ultimately improves the prediction performance. We take the AD semantic segmentation task as an example to assess the proposed MoE-RAM. Extensive experiments on AD datasets demonstrate the superiority of MoE-RAM compared to other MoE baselines and conventional single-model approaches.