F2SD: A dataset for end-to-end group detection algorithms

TL;DR

This paper introduces F2SD, a large-scale simulated image dataset for end-to-end group detection, enabling deep learning approaches directly from images, which was previously limited by lack of data and reliance on sensor signals.

Contribution

The paper presents a new large-scale simulated dataset F2SD and an end-to-end baseline model for F-formation detection from images, advancing the research methodology.

Findings

F2SD contains nearly 60,000 images from GTA-5 with annotations.

The proposed model demonstrates effective end-to-end group detection.

The dataset enables training deep learning models directly from images.

Abstract

The lack of large-scale datasets has been impeding the advance of deep learning approaches to the problem of F-formation detection. Moreover, most research works on this problem rely on input sensor signals of object location and orientation rather than image signals. To address this, we develop a new, large-scale dataset of simulated images for F-formation detection, called F-formation Simulation Dataset (F2SD). F2SD contains nearly 60,000 images simulated from GTA-5, with bounding boxes and orientation information on images, making it useful for a wide variety of modelling approaches. It is also closer to practical scenarios, where three-dimensional location and orientation information are costly to record. It is challenging to construct such a large-scale simulated dataset while keeping it realistic. Furthermore, the available research utilizes conventional methods to detect groups. They do not detect groups directly from the image. In this work, we propose (1) a large-scale simulation dataset F2SD and a pipeline for F-formation simulation, (2) a first-ever end-to-end baseline model for the task, and experiments on our simulation dataset.