# Tracking Gaze and Visual Focus of Attention of People Involved in Social Interaction

## TL;DR

This paper introduces a Bayesian switching state-space model to estimate and track visual focus of attention and gaze in multi-party social interactions, especially when eyes are not visible, using head movements and gaze correlation.

## Contribution

It presents a novel probabilistic model that jointly estimates gaze and VFOA without relying on eye detection, applicable to multi-party interactions.

## Key findings

- Effective tracking of gaze and VFOA demonstrated on public datasets.
- Outperforms existing methods in multi-party social interaction scenarios.
- Robust to situations where eyes are not visible.

## Abstract

The visual focus of attention (VFOA) has been recognized as a prominent conversational cue. We are interested in estimating and tracking the VFOAs associated with multi-party social interactions. We note that in this type of situations the participants either look at each other or at an object of interest; therefore their eyes are not always visible. Consequently both gaze and VFOA estimation cannot be based on eye detection and tracking. We propose a method that exploits the correlation between eye gaze and head movements. Both VFOA and gaze are modeled as latent variables in a Bayesian switching state-space model. The proposed formulation leads to a tractable learning procedure and to an efficient algorithm that simultaneously tracks gaze and visual focus. The method is tested and benchmarked using two publicly available datasets that contain typical multi-party human-robot and human-human interactions.