# Are you really looking at me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze from Conventional Video

## TL;DR

This paper presents ICE, a novel video-based framework for estimating interpersonal eye gaze without specialized hardware, validated across different settings, and demonstrating its usefulness in deception detection and communication skill assessment.

## Contribution

The paper introduces ICE, a new feature-extraction framework that automatically estimates interpersonal gaze from standard videos without prior participant location knowledge.

## Key findings

- ICE achieves high accuracy with infrared gaze tracker (F1=0.846).
- Interpersonal gaze correlates with deception and communication skills.
- Gaze alone outperforms facial expressions in predicting communication skills.

## Abstract

Despite a revolution in the pervasiveness of video cameras in our daily lives, one of the most meaningful forms of nonverbal affective communication, interpersonal eye gaze, i.e. eye gaze relative to a conversation partner, is not available from common video. We introduce the Interpersonal-Calibrating Eye-gaze Encoder (ICE), which automatically extracts interpersonal gaze from video recordings without specialized hardware and without prior knowledge of participant locations. Leveraging the intuition that individuals spend a large portion of a conversation looking at each other enables the ICE dynamic clustering algorithm to extract interpersonal gaze. We validate ICE in both video chat using an objective metric with an infrared gaze tracker (F1=0.846, N=8), as well as in face-to-face communication with expert-rated evaluations of eye contact (r= 0.37, N=170). We then use ICE to analyze behavior in two different, yet important affective communication domains: interrogation-based deception detection, and communication skill assessment in speed dating. We find that honest witnesses break interpersonal gaze contact and look down more often than deceptive witnesses when answering questions (p=0.004, d=0.79). In predicting expert communication skill ratings in speed dating videos, we demonstrate that interpersonal gaze alone has more predictive power than facial expressions.