IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline

TL;DR

IndoorCrowd is a comprehensive multi-scene indoor human dataset with annotations for detection, segmentation, and tracking, enabling evaluation of foundation-model auto-annotators and baseline methods across varied indoor environments.

Contribution

It introduces a large-scale, multi-scene indoor human dataset with automated annotation benchmarks and baseline evaluations, addressing the lack of real-world indoor complexity in existing datasets.

Findings

Auto-annotators achieve varying accuracy compared to human labels.

Detection, segmentation, and tracking baselines reveal scene-dependent difficulty.

Crowd density and occlusion significantly impact task performance.

Abstract

Understanding human behaviour in crowded indoor environments is central to surveillance, smart buildings, and human-robot interaction, yet existing datasets rarely capture real-world indoor complexity at scale. We introduce IndoorCrowd, a multi-scene dataset for indoor human detection, instance segmentation, and multi-object tracking, collected across four campus locations (ACS-EC, ACS-EG, IE-Central, R-Central). It comprises $31$ videos ($9{,}913$ frames at $5$fps) with human-verified, per-instance segmentation masks. A $620$-frame control subset benchmarks three foundation-model auto-annotators: SAM3, GroundingSAM, and EfficientGroundingSAM, against human labels using Cohen's $\kappa$, AP, precision, recall, and mask IoU. A further $2{,}552$-frame subset supports multi-object tracking with continuous identity tracks in MOTChallenge format. We establish detection, segmentation, and tracking baselines using YOLOv8n, YOLOv26n, and RT-DETR-L paired with ByteTrack, BoT-SORT, and OC-SORT. Per-scene analysis reveals substantial difficulty variation driven by crowd density, scale, and occlusion: ACS-EC, with $79.3\%$ dense frames and a mean instance scale of $60.8$px, is the most challenging scene. The project page is available at https://sheepseb.github.io/IndoorCrowd/.