OccFace: Unified Occlusion-Aware Facial Landmark Detection with Per-Point Visibility

TL;DR

OccFace is a novel facial landmark detection framework that explicitly predicts per-point visibility, improving robustness under occlusion and large head rotations across diverse face types.

Contribution

It introduces a unified dense 100-point landmark layout with an occlusion module that jointly predicts landmarks and visibility, enhancing occlusion handling in facial landmark detection.

Findings

Improved accuracy on occluded landmarks

Enhanced robustness under external occlusion and rotations

Benchmarking with new occlusion-aware evaluation suite

Abstract

Accurate facial landmark detection under occlusion remains challenging, especially for human-like faces with large appearance variation and rotation-driven self-occlusion. Existing detectors typically localize landmarks while handling occlusion implicitly, without predicting per-point visibility that downstream applications can benefits. We present OccFace, an occlusion-aware framework for universal human-like faces, including humans, stylized characters, and other non-human designs. OccFace adopts a unified dense 100-point layout and a heatmap-based backbone, and adds an occlusion module that jointly predicts landmark coordinates and per-point visibility by combining local evidence with cross-landmark context. Visibility supervision mixes manual labels with landmark-aware masking that derives pseudo visibility from mask-heatmap overlap. We also create an occlusion-aware evaluation suite reporting NME on visible vs. occluded landmarks and benchmarking visibility with Occ AP, F1@0.5, and ROC-AUC, together with a dataset annotated with 100-point landmarks and per-point visibility. Experiments show improved robustness under external occlusion and large head rotations, especially on occluded regions, while preserving accuracy on visible landmarks.