GazeDETR: Gaze Detection using Disentangled Head and Gaze Representations

TL;DR

GazeDETR introduces a novel end-to-end gaze detection model with disentangled decoders for head and gaze, achieving state-of-the-art results by effectively leveraging local and global information.

Contribution

The paper presents a new architecture with separate decoders for head and gaze, improving gaze detection accuracy over existing end-to-end models.

Findings

Achieves state-of-the-art results on multiple datasets.

Outperforms existing models with a significant margin.

Utilizes local and global information effectively.

Abstract

Gaze communication plays a crucial role in daily social interactions. Quantifying this behavior can help in human-computer interaction and digital phenotyping. While end-to-end models exist for gaze target detection, they only utilize a single decoder to simultaneously localize human heads and predict their corresponding gaze (e.g., 2D points or heatmap) in a scene. This multitask learning approach generates a unified and entangled representation for human head localization and gaze location prediction. Herein, we propose GazeDETR, a novel end-to-end architecture with two disentangled decoders that individually learn unique representations and effectively utilize coherent attentive fields for each subtask. More specifically, we demonstrate that its human head predictor utilizes local information, while its gaze decoder incorporates both local and global information. Our proposed architecture achieves state-of-the-art results on the GazeFollow, VideoAttentionTarget and ChildPlay datasets. It outperforms existing end-to-end models with a notable margin.