Fast Multiple Landmark Localisation Using a Patch-based Iterative Network

TL;DR

This paper introduces a fast, efficient patch-based iterative neural network for accurate 3D landmark localization in medical images, combining regression and classification, and extending to multiple landmarks with global anatomical modeling.

Contribution

The novel Patch-based Iterative Network (PIN) achieves rapid and accurate landmark localization in 3D medical images using an iterative CNN approach with multi-task learning and global landmark modeling.

Findings

Average localization error of 5.59mm

Runtime of 0.44 seconds for 10 landmarks

Qualitative similarity to clinical ground truth

Abstract

We propose a new Patch-based Iterative Network (PIN) for fast and accurate landmark localisation in 3D medical volumes. PIN utilises a Convolutional Neural Network (CNN) to learn the spatial relationship between an image patch and anatomical landmark positions. During inference, patches are repeatedly passed to the CNN until the estimated landmark position converges to the true landmark location. PIN is computationally efficient since the inference stage only selectively samples a small number of patches in an iterative fashion rather than a dense sampling at every location in the volume. Our approach adopts a multi-task learning framework that combines regression and classification to improve localisation accuracy. We extend PIN to localise multiple landmarks by using principal component analysis, which models the global anatomical relationships between landmarks. We have evaluated PIN using 72 3D ultrasound images from fetal screening examinations. PIN achieves quantitatively an average landmark localisation error of 5.59mm and a runtime of 0.44s to predict 10 landmarks per volume. Qualitatively, anatomical 2D standard scan planes derived from the predicted landmark locations are visually similar to the clinical ground truth. Source code is publicly available at https://github.com/yuanwei1989/landmark-detection.