Multi-view Gaze Target Estimation

TL;DR

This paper introduces a multi-view gaze target estimation method that leverages multiple camera views to improve accuracy, handle occlusions, and estimate gaze targets even when the face is not visible, supported by a new dataset.

Contribution

The paper proposes a novel multi-view GTE approach with modules for head information aggregation, uncertainty-based gaze selection, and scene attention, along with a new dataset for evaluation.

Findings

Outperforms single-view baselines significantly

Enables gaze estimation even with occluded faces

Provides a new dataset for multi-view GTE

Abstract

This paper presents a method that utilizes multiple camera views for the gaze target estimation (GTE) task. The approach integrates information from different camera views to improve accuracy and expand applicability, addressing limitations in existing single-view methods that face challenges such as face occlusion, target ambiguity, and out-of-view targets. Our method processes a pair of camera views as input, incorporating a Head Information Aggregation (HIA) module for leveraging head information from both views for more accurate gaze estimation, an Uncertainty-based Gaze Selection (UGS) for identifying the most reliable gaze output, and an Epipolar-based Scene Attention (ESA) module for cross-view background information sharing. This approach significantly outperforms single-view baselines, especially when the second camera provides a clear view of the person's face. Additionally, our method can estimate the gaze target in the first view using the image of the person in the second view only, a capability not possessed by single-view GTE methods. Furthermore, the paper introduces a multi-view dataset for developing and evaluating multi-view GTE methods. Data and code are available at https://www3.cs.stonybrook.edu/~cvl/multiview_gte.html