Cranio-ID: Graph-Based Craniofacial Identification via Automatic Landmark Annotation in 2D Multi-View X-rays

TL;DR

This paper introduces Cranio-ID, a novel graph-based framework for automatic craniofacial landmark annotation and cross-modal matching in 2D X-ray skulls, improving reliability and accuracy in forensic identification.

Contribution

It presents a new method combining YOLO-pose landmark detection with graph-based cross-modal matching using cross-attention and optimal transport, validated on multiple datasets.

Findings

Significant accuracy improvements in landmark annotation.

Enhanced reliability in skull-to-face and sketch-to-face matching.

Validated effectiveness across diverse datasets.

Abstract

In forensic craniofacial identification and in many biomedical applications, craniometric landmarks are important. Traditional methods for locating landmarks are time-consuming and require specialized knowledge and expertise. Current methods utilize superimposition and deep learning-based methods that employ automatic annotation of landmarks. However, these methods are not reliable due to insufficient large-scale validation studies. In this paper, we proposed a novel framework Cranio-ID: First, an automatic annotation of landmarks on 2D skulls (which are X-ray scans of faces) with their respective optical images using our trained YOLO-pose models. Second, cross-modal matching by formulating these landmarks into graph representations and then finding semantic correspondence between graphs of these two modalities using cross-attention and optimal transport framework. Our proposed framework is validated on the S2F and CUHK datasets (CUHK dataset resembles with S2F dataset). Extensive experiments have been conducted to evaluate the performance of our proposed framework, which demonstrates significant improvements in both reliability and accuracy, as well as its effectiveness in cross-domain skull-to-face and sketch-to-face matching in forensic science.