ChildPlay: A New Benchmark for Understanding Children's Gaze Behaviour

TL;DR

This paper introduces ChildPlay, a novel dataset and model for predicting children's gaze behavior in natural settings, addressing the gap in existing benchmarks focused on adults and lab environments.

Contribution

It provides the first dataset and model specifically designed for understanding gaze behavior in children and interacting adults in uncontrolled environments.

Findings

The model achieves state-of-the-art results on ChildPlay and existing benchmarks.

Gaze prediction on children is less accurate than on adults but improves with fine-tuning.

Child-specific gaze models outperform adult-trained models when adapted to children.

Abstract

Gaze behaviors such as eye-contact or shared attention are important markers for diagnosing developmental disorders in children. While previous studies have looked at some of these elements, the analysis is usually performed on private datasets and is restricted to lab settings. Furthermore, all publicly available gaze target prediction benchmarks mostly contain instances of adults, which makes models trained on them less applicable to scenarios with young children. In this paper, we propose the first study for predicting the gaze target of children and interacting adults. To this end, we introduce the ChildPlay dataset: a curated collection of short video clips featuring children playing and interacting with adults in uncontrolled environments (e.g. kindergarten, therapy centers, preschools etc.), which we annotate with rich gaze information. We further propose a new model for gaze target prediction that is geometrically grounded by explicitly identifying the scene parts in the 3D field of view (3DFoV) of the person, leveraging recent geometry preserving depth inference methods. Our model achieves state of the art results on benchmark datasets and ChildPlay. Furthermore, results show that looking at faces prediction performance on children is much worse than on adults, and can be significantly improved by fine-tuning models using child gaze annotations. Our dataset and models will be made publicly available.