F-formation Detection: Individuating Free-standing Conversational Groups in Images

TL;DR

This paper introduces a new graph-cuts based method, GCFF, for detecting social groups in images by modeling F-formations, outperforming existing methods in accuracy, robustness, and versatility.

Contribution

The paper provides a formal social science-based definition of groups and develops a novel graph-cuts algorithm for F-formation detection in images.

Findings

GCFF outperforms state-of-the-art methods in accuracy

Demonstrates robustness to noise

Effective in recognizing groups of various sizes

Abstract

Detection of groups of interacting people is a very interesting and useful task in many modern technologies, with application fields spanning from video-surveillance to social robotics. In this paper we first furnish a rigorous definition of group considering the background of the social sciences: this allows us to specify many kinds of group, so far neglected in the Computer Vision literature. On top of this taxonomy, we present a detailed state of the art on the group detection algorithms. Then, as a main contribution, we present a brand new method for the automatic detection of groups in still images, which is based on a graph-cuts framework for clustering individuals; in particular we are able to codify in a computational sense the sociological definition of F-formation, that is very useful to encode a group having only proxemic information: position and orientation of people. We call the proposed method Graph-Cuts for F-formation (GCFF). We show how GCFF definitely outperforms all the state of the art methods in terms of different accuracy measures (some of them are brand new), demonstrating also a strong robustness to noise and versatility in recognizing groups of various cardinality.