High-Fidelity Human Avatars from Laptop Webcams using Edge Compute

TL;DR

This paper presents a method to generate high-fidelity, animatable human avatars from low-quality laptop webcams using edge computing on consumer devices.

Contribution

It introduces a novel system that combines 3D models, GANs, and differentiable rendering to produce realistic avatars on limited on-device compute.

Findings

Successfully generates photo-realistic avatars from low-res webcam images.

Operates efficiently on AMD mobile processors without offloading to servers.

Enables real-time avatar creation suitable for video conferencing.

Abstract

Photo-realistic human avatars have broad applications, yet high-fidelity avatar generation has traditionally required expensive professional camera rigs and extensive artistic labor. Recent research has enabled constructing them automatically from smartphones with RGB and IR sensors, however, these new methods still rely on high-resolution cameras on modern smartphones and often require offloading the processing to powerful servers with GPUs. Modern applications such as video conferencing call for the ability to generate these avatars from consumer-grade laptop webcams using limited compute available on-device. In this work, we develop a novel method based on 3D morphable models, landmark detection, photorealistic texture GANs, and differentiable rendering to tackle the problem of low webcam image quality and edge computation. We build an automatic system to generate high-fidelity animatable avatars under these limitations, leveraging the compute capabilities of AMD mobile processors.