Differentiable Nonparametric Belief Propagation

TL;DR

This paper introduces a differentiable nonparametric belief propagation method that learns probabilistic factors with neural networks, improving articulated pose tracking by replacing handcrafted features with learned ones through end-to-end training.

Contribution

The work develops a differentiable framework for nonparametric belief propagation, enabling learned probabilistic factors via neural networks, which enhances tracking performance over traditional handcrafted methods.

Findings

Learned factors improve pose tracking accuracy.

End-to-end training effectively optimizes probabilistic models.

Outperforms recurrent neural networks in experiments.

Abstract

We present a differentiable approach to learn the probabilistic factors used for inference by a nonparametric belief propagation algorithm. Existing nonparametric belief propagation methods rely on domain-specific features encoded in the probabilistic factors of a graphical model. In this work, we replace each crafted factor with a differentiable neural network enabling the factors to be learned using an efficient optimization routine from labeled data. By combining differentiable neural networks with an efficient belief propagation algorithm, our method learns to maintain a set of marginal posterior samples using end-to-end training. We evaluate our differentiable nonparametric belief propagation (DNBP) method on a set of articulated pose tracking tasks and compare performance with a recurrent neural network. Results from this comparison demonstrate the effectiveness of using learned factors for tracking and suggest the practical advantage over hand-crafted approaches. The project webpage is available at: progress.eecs.umich.edu/projects/dnbp.