RTGaze: Real-Time 3D-Aware Gaze Redirection from a Single Image
Hengfei Wang, Zhongqun Zhang, Yihua Cheng, Hyung Jin Chang

TL;DR
RTGaze is a real-time, 3D-aware gaze redirection method that produces high-quality, controllable face images from a single image, significantly outperforming previous methods in speed and accuracy.
Contribution
This work introduces RTGaze, a novel neural rendering approach that achieves real-time, high-quality gaze redirection with 3D consistency from a single face image.
Findings
RTGaze operates at approximately 0.06 seconds per image.
It outperforms previous methods in redirection accuracy and image quality.
RTGaze is 800 times faster than prior 3D-aware gaze redirection techniques.
Abstract
Gaze redirection methods aim to generate realistic human face images with controllable eye movement. However, recent methods often struggle with 3D consistency, efficiency, or quality, limiting their practical applications. In this work, we propose RTGaze, a real-time and high-quality gaze redirection method. Our approach learns a gaze-controllable facial representation from face images and gaze prompts, then decodes this representation via neural rendering for gaze redirection. Additionally, we distill face geometric priors from a pretrained 3D portrait generator to enhance generation quality. We evaluate RTGaze both qualitatively and quantitatively, demonstrating state-of-the-art performance in efficiency, redirection accuracy, and image quality across multiple datasets. Our system achieves real-time, 3D-aware gaze redirection with a feedforward network (~0.06 sec/image), making it…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsFacial Nerve Paralysis Treatment and Research · Face recognition and analysis · Gaze Tracking and Assistive Technology
