High-quality Animatable Eyelid Shapes from Lightweight Captures

TL;DR

This paper introduces a novel method for high-quality eyelid reconstruction and animation using only lightweight RGB video captures, leveraging eyeball information and neural control for realistic results.

Contribution

It presents the first approach for detailed eyelid reconstruction and animation from lightweight RGB video captures, integrating eyeball parameters and neural control modules.

Findings

Outperforms previous methods in detail and realism

Works effectively on synthetic and real data

Uses only mobile phone RGB videos for capture

Abstract

High-quality eyelid reconstruction and animation are challenging for the subtle details and complicated deformations. Previous works usually suffer from the trade-off between the capture costs and the quality of details. In this paper, we propose a novel method that can achieve detailed eyelid reconstruction and animation by only using an RGB video captured by a mobile phone. Our method utilizes both static and dynamic information of eyeballs (e.g., positions and rotations) to assist the eyelid reconstruction, cooperating with an automatic eyeball calibration method to get the required eyeball parameters. Furthermore, we develop a neural eyelid control module to achieve the semantic animation control of eyelids. To the best of our knowledge, we present the first method for high-quality eyelid reconstruction and animation from lightweight captures. Extensive experiments on both synthetic and real data show that our method can provide more detailed and realistic results compared with previous methods based on the same-level capture setups. The code is available at https://github.com/StoryMY/AniEyelid.