REFORM: Recognizing F-formations for Social Robots

TL;DR

This paper presents REFORM, a machine learning-based method for detecting F-formations in social interactions, improving accuracy over previous heuristic algorithms by analyzing human and robot positional data.

Contribution

The paper introduces a novel data-driven approach, REFORM, for recognizing F-formations that outperforms existing heuristic methods and includes new quantitative measures for F-formation characterization.

Findings

REFORM achieves higher accuracy than state-of-the-art algorithms.

The approach is validated on three diverse datasets, including human and robot scenarios.

New measures of symmetry and tightness effectively characterize F-formations.

Abstract

Recognizing and understanding conversational groups, or F-formations, is a critical task for situated agents designed to interact with humans. F-formations contain complex structures and dynamics, yet are used intuitively by people in everyday face-to-face conversations. Prior research exploring ways of identifying F-formations has largely relied on heuristic algorithms that may not capture the rich dynamic behaviors employed by humans. We introduce REFORM (REcognize F-FORmations with Machine learning), a data-driven approach for detecting F-formations given human and agent positions and orientations. REFORM decomposes the scene into all possible pairs and then reconstructs F-formations with a voting-based scheme. We evaluated our approach across three datasets: the SALSA dataset, a newly collected human-only dataset, and a new set of acted human-robot scenarios, and found that REFORM yielded improved accuracy over a state-of-the-art F-formation detection algorithm. We also introduce symmetry and tightness as quantitative measures to characterize F-formations. Supplementary video: https://youtu.be/Fp7ETdkKvdA , Dataset available at: github.com/cu-ironlab/Babble