All's well that FID's well? Result quality and metric scores in GAN models for lip-sychronization tasks

TL;DR

This paper evaluates the performance of GAN models for lip-synchronization, comparing LipGAN and a new variation L1WGAN-GP on the GRID dataset to assess result quality and metric scores.

Contribution

The study reimplements LipGAN in PyTorch and introduces L1WGAN-GP, a novel variation tailored for lip-synchronization tasks, providing a comparative analysis.

Findings

L1WGAN-GP outperforms LipGAN in certain metrics

Reimplementation in PyTorch ensures reproducibility

Comparison highlights strengths and weaknesses of each model

Abstract

We test the performance of GAN models for lip-synchronization. For this, we reimplement LipGAN in Pytorch, train it on the dataset GRID and compare it to our own variation, L1WGAN-GP, adapted to the LipGAN architecture and also trained on GRID.