Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies

TL;DR

This paper introduces a deep learning approach that combines multi-view feature fusion and attention mechanisms to enhance kidney stone recognition accuracy from endoscopic images.

Contribution

It proposes a novel multi-view feature fusion method with attention layers, improving kidney stone classification performance over existing techniques.

Findings

Attention layers increased single view accuracy by 4%.

Multi-view fusion improved overall classification accuracy by up to 11%.

Method mimics biological analysis for better feature discrimination.

Abstract

This contribution presents a deep learning method for the extraction and fusion of information relating to kidney stone fragments acquired from different viewpoints of the endoscope. Surface and section fragment images are jointly used during the training of the classifier to improve the discrimination power of the features by adding attention layers at the end of each convolutional block. This approach is specifically designed to mimic the morpho-constitutional analysis performed in ex-vivo by biologists to visually identify kidney stones by inspecting both views. The addition of attention mechanisms to the backbone improved the results of single view extraction backbones by 4% on average. Moreover, in comparison to the state-of-the-art, the fusion of the deep features improved the overall results up to 11% in terms of kidney stone classification accuracy.