DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs

TL;DR

DualSMC introduces a novel framework combining adversarial particle filtering and SMC-based planning to effectively solve continuous POMDPs with complex observations, maintaining interpretability and handling uncertainty.

Contribution

It proposes the DualSMC network that integrates adversarial filtering and trajectory planning for continuous POMDPs, a novel approach in this domain.

Findings

Effective in three continuous POMDP domains.

Handles complex observations like images.

Maintains high interpretability.

Abstract

A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty. We cast POMDP filtering and planning problems as two closely related Sequential Monte Carlo (SMC) processes, one over the real states and the other over the future optimal trajectories, and combine the merits of these two parts in a new model named the DualSMC network. In particular, we first introduce an adversarial particle filter that leverages the adversarial relationship between its internal components. Based on the filtering results, we then propose a planning algorithm that extends the previous SMC planning approach [Piche et al., 2018] to continuous POMDPs with an uncertainty-dependent policy. Crucially, not only can DualSMC handle complex observations such as image input but also it remains highly interpretable. It is shown to be effective in three continuous POMDP domains: the floor positioning domain, the 3D light-dark navigation domain, and a modified Reacher domain.