Two-Stream Deep Feature Modelling for Automated Video Endoscopy Data Analysis

TL;DR

This paper introduces a two-stream deep learning model with a relational network for automated analysis of endoscopic GI images, improving classification accuracy over existing methods.

Contribution

The paper presents a novel two-stream deep feature model with a relational network architecture for endoscopic image analysis, outperforming state-of-the-art methods.

Findings

Outperforms existing methods on KVASIR and Nerthus datasets

Two-stream input significantly improves classification accuracy

Relational network effectively combines feature streams

Abstract

Automating the analysis of imagery of the Gastrointestinal (GI) tract captured during endoscopy procedures has substantial potential benefits for patients, as it can provide diagnostic support to medical practitioners and reduce mistakes via human error. To further the development of such methods, we propose a two-stream model for endoscopic image analysis. Our model fuses two streams of deep feature inputs by mapping their inherent relations through a novel relational network model, to better model symptoms and classify the image. In contrast to handcrafted feature-based models, our proposed network is able to learn features automatically and outperforms existing state-of-the-art methods on two public datasets: KVASIR and Nerthus. Our extensive evaluations illustrate the importance of having two streams of inputs instead of a single stream and also demonstrates the merits of the proposed relational network architecture to combine those streams.