Generative Landmarks

TL;DR

This paper introduces a generative adversarial network approach for landmark detection that enhances temporal consistency and personalization without requiring manual annotations, applicable across different image classes like faces and hands.

Contribution

It presents a novel, annotation-free landmark detection method using image translation and cyclic consistency, improving consistency and personalization.

Findings

Achieves temporally consistent landmark detection

Does not rely on manual landmark annotations

Works across multiple image classes

Abstract

We propose a general purpose approach to detect landmarks with improved temporal consistency, and personalization. Most sparse landmark detection methods rely on laborious, manually labelled landmarks, where inconsistency in annotations over a temporal volume leads to sub-optimal landmark learning. Further, high-quality landmarks with personalization is often hard to achieve. We pose landmark detection as an image translation problem. We capture two sets of unpaired marked (with paint) and unmarked videos. We then use a generative adversarial network and cyclic consistency to predict deformations of landmark templates that simulate markers on unmarked images until these images are indistinguishable from ground-truth marked images. Our novel method does not rely on manually labelled priors, is temporally consistent, and image class agnostic -- face, and hand landmarks detection examples are shown.