EyeBAG: Accurate Control of Eye Blink and Gaze Based on Data Augmentation Leveraging Style Mixing

TL;DR

This paper introduces EyeBAG, a framework with style mixing-based data augmentation for precise eye blink and gaze control in face images, enhancing downstream task performance.

Contribution

The paper presents a novel framework with dedicated modules for eye blink and gaze control, utilizing style mixing for data augmentation to improve control accuracy.

Findings

High-quality eye-controlled face images generated

Improved performance in downstream face analysis tasks

Effective style mixing data augmentation method

Abstract

Recent developments in generative models have enabled the generation of photo-realistic human face images, and downstream tasks utilizing face generation technology have advanced accordingly. However, models for downstream tasks are yet substandard at eye control (e.g. eye blink, gaze redirection). To overcome such eye control problems, we introduce a novel framework consisting of two distinct modules: a blink control module and a gaze redirection module. We also propose a novel data augmentation method to train each module, leveraging style mixing to obtain images with desired features. We show that our framework produces eye-controlled images of high quality, and demonstrate how it can be used to improve the performance of downstream tasks.