# Not Only Look But Observe: Variational Observation Model of Scene-Level 3D Multi-Object Understanding for Probabilistic SLAM

## TL;DR

This paper introduces NOLBO, a variational observation model that estimates scene-level 3D multi-object understanding from 2D images, enhancing probabilistic SLAM by modeling object relations and scene context.

## Contribution

It proposes a novel variational auto-encoder based approach to approximate the Bayesian observation model for scene-level multi-object 3D understanding from single shots.

## Key findings

- Enables probabilistic inference with object relations and scene context.
- Estimates 3D shape and pose from 2D images using latent variables.
- Facilitates object-oriented data association in SLAM.

## Abstract

We present NOLBO, a variational observation model estimation for 3D multi-object from 2D single shot. Previous probabilistic instance-level understandings mainly consider the single-object image, not single shot with multi-object; relations between objects and the entire scene are out of their focus. The objectness of each observation also hardly join their model. Therefore, we propose a method to approximate the Bayesian observation model of scene-level 3D multi-object understanding. By exploiting variational auto-encoder (VAE), we estimate latent variables from the entire scene, which follow tractable distributions and concurrently imply 3D full shape and pose. To perform object-oriented data association and probabilistic simultaneous localization and mapping (SLAM), our observation models can easily be adopted to probabilistic inference by replacing object-oriented features with latent variables.