TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

TL;DR

TO-Scene introduces a large-scale, diverse dataset of 3D tabletop scenes created through a scalable crowdsourcing framework, enabling improved understanding of small tabletop objects in indoor scene parsing.

Contribution

The paper presents TO-Scene, a novel large-scale dataset for 3D tabletop scenes, along with a tabletop-aware learning strategy that enhances scene understanding tasks.

Findings

Algorithms trained on TO-Scene perform well on real scanned data.

The tabletop-aware strategy significantly improves 3D segmentation and detection results.

TO-Scene facilitates better understanding of small tabletop objects in indoor scenes.

Abstract

Many basic indoor activities such as eating or writing are always conducted upon different tabletops (e.g., coffee tables, writing desks). It is indispensable to understanding tabletop scenes in 3D indoor scene parsing applications. Unfortunately, it is hard to meet this demand by directly deploying data-driven algorithms, since 3D tabletop scenes are rarely available in current datasets. To remedy this defect, we introduce TO-Scene, a large-scale dataset focusing on tabletop scenes, which contains 20,740 scenes with three variants. To acquire the data, we design an efficient and scalable framework, where a crowdsourcing UI is developed to transfer CAD objects from ModelNet and ShapeNet onto tables from ScanNet, then the output tabletop scenes are simulated into real scans and annotated automatically. Further, a tabletop-aware learning strategy is proposed for better perceiving the small-sized tabletop instances. Notably, we also provide a real scanned test set TO-Real to verify the practical value of TO-Scene. Experiments show that the algorithms trained on TO-Scene indeed work on the realistic test data, and our proposed tabletop-aware learning strategy greatly improves the state-of-the-art results on both 3D semantic segmentation and object detection tasks. Dataset and code are available at https://github.com/GAP-LAB-CUHK-SZ/TO-Scene.