Domain-Specialized Object Detection via Model-Level Mixtures of Experts

TL;DR

This paper introduces a model-level Mixture-of-Experts architecture for object detection that combines YOLO-based detectors trained on different data subsets, improving performance and interpretability over standard ensembles.

Contribution

It proposes a novel MoE framework for object detection with a learned gating network and demonstrates its effectiveness on the BDD100K dataset.

Findings

MoE outperforms standard ensemble methods in object detection tasks.

The approach provides insights into expert specialization across different domains.

The proposed method enhances interpretability of object detection models.

Abstract

Mixture-of-Experts (MoE) models provide a structured approach to combining specialized neural networks and offer greater interpretability than conventional ensembles. While MoEs have been successfully applied to image classification and semantic segmentation, their use in object detection remains limited due to challenges in merging dense and structured predictions. In this work, we investigate model-level mixtures of object detectors and analyze their suitability for improving performance and interpretability in object detection. We propose an MoE architecture that combines YOLO-based detectors trained on semantically disjoint data subsets, with a learned gating network that dynamically weights expert contributions. We study different strategies for fusing detection outputs and for training the gating mechanism, including balancing losses to prevent expert collapse. Experiments on the BDD100K dataset demonstrate that the proposed MoE consistently outperforms standard ensemble approaches and provides insights into expert specialization across domains, highlighting model-level MoEs as a viable alternative to traditional ensembling for object detection. Our code is available at https://github.com/KASTEL-MobilityLab/mixtures-of-experts/.