Monte Carlo Scene Search for 3D Scene Understanding
Shreyas Hampali, Sinisa Stekovic, Sayan Deb Sarkar, Chetan Srinivasa, Kumar, Friedrich Fraundorfer, Vincent Lepetit

TL;DR
This paper introduces a modified Monte Carlo Tree Search algorithm for 3D scene understanding from noisy RGB-D data, enabling efficient analysis-by-synthesis without extensive training data, and demonstrates superior layout retrieval on ScanNet.
Contribution
It adapts MCTS for perception tasks, reducing reliance on training data and hyperparameter tuning, and improves 3D scene analysis from RGB-D scans.
Findings
Often retrieves configurations better than manual annotations.
Effective in optimizing object and layout hypotheses.
Demonstrates applicability on the ScanNet dataset.
Abstract
We explore how a general AI algorithm can be used for 3D scene understanding to reduce the need for training data. More exactly, we propose a modification of the Monte Carlo Tree Search (MCTS) algorithm to retrieve objects and room layouts from noisy RGB-D scans. While MCTS was developed as a game-playing algorithm, we show it can also be used for complex perception problems. Our adapted MCTS algorithm has few easy-to-tune hyperparameters and can optimise general losses. We use it to optimise the posterior probability of objects and room layout hypotheses given the RGB-D data. This results in an analysis-by-synthesis approach that explores the solution space by rendering the current solution and comparing it to the RGB-D observations. To perform this exploration even more efficiently, we propose simple changes to the standard MCTS' tree construction and exploration policy. We…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Topics3D Shape Modeling and Analysis · Robotics and Sensor-Based Localization · Advanced Vision and Imaging
MethodsMonte-Carlo Tree Search
