GazeOnce360: Fisheye-Based 360{\deg} Multi-Person Gaze Estimation with Global-Local Feature Fusion

TL;DR

GazeOnce360 introduces an end-to-end fisheye-based model for 3D multi-person gaze estimation in 360-degree scenes, utilizing a new synthetic dataset and a dual-resolution architecture to handle distortion and capture eye details.

Contribution

The paper presents a novel approach for multi-person gaze estimation from fisheye images, including a large synthetic dataset and a dual-resolution model with feature fusion.

Findings

Effective handling of fisheye distortion through rotational convolutions.

Improved gaze estimation accuracy with dual-resolution feature fusion.

Demonstrated feasibility of 360-degree multi-person gaze estimation.

Abstract

We present GazeOnce360, a novel end-to-end model for multi-person gaze estimation from a single tabletop-mounted upward-facing fisheye camera. Unlike conventional approaches that rely on forward-facing cameras in constrained viewpoints, we address the underexplored setting of estimating the 3D gaze direction of multiple people distributed across a 360{\deg} scene from an upward fisheye perspective. To support research in this setting, we introduce MPSGaze360, a large-scale synthetic dataset rendered using Unreal Engine, featuring diverse multi-person configurations with accurate 3D gaze and eye landmark annotations. Our model tackles the severe distortion and perspective variation inherent in fisheye imagery by incorporating rotational convolutions and eye landmark supervision. To better capture fine-grained eye features crucial for gaze estimation, we propose a dual-resolution architecture that fuses global low-resolution context with high-resolution local eye regions. Experimental results demonstrate the effectiveness of each component in our model. This work highlights the feasibility and potential of fisheye-based 360{\deg} gaze estimation in practical multi-person scenarios. Project page: https://caizhuojiang.github.io/GazeOnce360/.