Diffusion Policy with Bayesian Expert Selection for Active Multi-Target Tracking

TL;DR

This paper introduces a Bayesian expert selection framework for diffusion policies in active multi-target tracking, improving decision-making by quantifying uncertainty and selecting strategies with the best worst-case predicted performance.

Contribution

It formulates expert selection as an offline contextual bandit problem using a Bayesian model, enabling uncertainty-aware strategy choice in diffusion policies.

Findings

Outperforms base diffusion policies in simulated indoor tracking scenarios.

Uses a multi-head VBLL model to predict expert performance with uncertainty.

Employs a Lower Confidence Bound criterion for robust expert selection.

Abstract

Active multi-target tracking requires a mobile robot to balance exploration for undetected targets with exploitation of uncertain tracked ones. Diffusion policies have emerged as a powerful approach for capturing diverse behavioral strategies by learning action sequences from expert demonstrations. However, existing methods implicitly select among strategies through the denoising process, without uncertainty quantification over which strategy to execute. We formulate expert selection for diffusion policies as an offline contextual bandit problem and propose a Bayesian framework for pessimistic, uncertainty-aware strategy selection. A multi-head Variational Bayesian Last Layer (VBLL) model predicts the expected tracking performance of each expert strategy given the current belief state, providing both a point estimate and predictive uncertainty. Following the pessimism principle for offline decision-making, a Lower Confidence Bound (LCB) criterion then selects the expert whose worst-case predicted performance is best, avoiding overcommitment to experts with unreliable predictions. The selected expert conditions a diffusion policy to generate corresponding action sequences. Experiments on simulated indoor tracking scenarios demonstrate that our approach outperforms both the base diffusion policy and standard gating methods, including Mixture-of-Experts selection and deterministic regression baselines.