# Real-time Approximate Bayesian Computation for Scene Understanding

## TL;DR

This paper presents a real-time approximate Bayesian computation framework for scene understanding tasks, leveraging realistic simulators and novel acceleration techniques to achieve fast and accurate inference in complex real-world scenarios.

## Contribution

It introduces a Bayesian inference approach using off-the-shelf simulators, combined with neural surrogates and adaptive discretization for real-time scene understanding.

## Key findings

- Achieves real-time performance on complex scene understanding tasks.
- Demonstrates improved inference speed with neural surrogates and adaptive discretization.
- Validates approach on real-world data with accurate results.

## Abstract

Consider scene understanding problems such as predicting where a person is probably reaching, or inferring the pose of 3D objects from depth images, or inferring the probable street crossings of pedestrians at a busy intersection. This paper shows how to solve these problems using Approximate Bayesian Computation. The underlying generative models are built from realistic simulation software, wrapped in a Bayesian error model for the gap between simulation outputs and real data. The simulators are drawn from off-the-shelf computer graphics, video game, and traffic simulation code. The paper introduces two techniques for speeding up inference that can be used separately or in combination. The first is to train neural surrogates of the simulators, using a simple form of domain randomization to make the surrogates more robust to the gap between the simulation and reality. The second is to adaptively discretize the latent variables using a Tree-pyramid approach adapted from computer graphics. This paper also shows performance and accuracy measurements on real-world problems, establishing that it is feasible to solve these problems in real-time.