DVGaze: Dual-View Gaze Estimation

TL;DR

This paper introduces DV-Gaze, a dual-view gaze estimation network that leverages dual-camera images and innovative modules to improve accuracy over single-view methods, achieving state-of-the-art results.

Contribution

The paper proposes a novel dual-view gaze estimation network with a dual-view interactive convolution block and a dual-view transformer, enhancing gaze estimation accuracy using dual-camera images.

Findings

DV-Gaze outperforms existing methods on ETH-XGaze and EVE datasets.

Dual-view features improve gaze estimation accuracy.

The dual-view transformer effectively encodes camera pose and geometric relations.

Abstract

Gaze estimation methods estimate gaze from facial appearance with a single camera. However, due to the limited view of a single camera, the captured facial appearance cannot provide complete facial information and thus complicate the gaze estimation problem. Recently, camera devices are rapidly updated. Dual cameras are affordable for users and have been integrated in many devices. This development suggests that we can further improve gaze estimation performance with dual-view gaze estimation. In this paper, we propose a dual-view gaze estimation network (DV-Gaze). DV-Gaze estimates dual-view gaze directions from a pair of images. We first propose a dual-view interactive convolution (DIC) block in DV-Gaze. DIC blocks exchange dual-view information during convolution in multiple feature scales. It fuses dual-view features along epipolar lines and compensates for the original feature with the fused feature. We further propose a dual-view transformer to estimate gaze from dual-view features. Camera poses are encoded to indicate the position information in the transformer. We also consider the geometric relation between dual-view gaze directions and propose a dual-view gaze consistency loss for DV-Gaze. DV-Gaze achieves state-of-the-art performance on ETH-XGaze and EVE datasets. Our experiments also prove the potential of dual-view gaze estimation. We release codes in https://github.com/yihuacheng/DVGaze.