Robust Image Descriptors for Real-Time Inter-Examination Retargeting in Gastrointestinal Endoscopy

TL;DR

This paper introduces a robust, real-time image descriptor for inter-examination retargeting in gastrointestinal endoscopy, enabling accurate matching despite tissue appearance changes over time.

Contribution

It presents a novel hierarchical intensity comparison descriptor combined with a binary coding scheme using random forests, improving matching robustness and speed for serial endoscopic images.

Findings

Outperforms state-of-the-art methods in validation tests

Enables real-time tissue matching over long-term intervals

Demonstrates robustness to tissue appearance changes

Abstract

For early diagnosis of malignancies in the gastrointestinal tract, surveillance endoscopy is increasingly used to monitor abnormal tissue changes in serial examinations of the same patient. Despite successes with optical biopsy for in vivo and in situ tissue characterisation, biopsy retargeting for serial examinations is challenging because tissue may change in appearance between examinations. In this paper, we propose an inter-examination retargeting framework for optical biopsy, based on an image descriptor designed for matching between endoscopic scenes over significant time intervals. Each scene is described by a hierarchy of regional intensity comparisons at various scales, offering tolerance to long-term change in tissue appearance whilst remaining discriminative. Binary coding is then used to compress the descriptor via a novel random forests approach, providing fast comparisons in Hamming space and real-time retargeting. Extensive validation conducted on 13 in vivo gastrointestinal videos, collected from six patients, show that our approach outperforms state-of-the-art methods.