Enhancing colorectal polyp classification using gaze-based attention networks
Zhenghao Guo, Yanyan Hu, Peixuan Ge, In Neng Chan, Tao Yan, Pak Kin Wong, Shaoyong Xu, Zheng Li, Shan Gao
TL;DR
This paper introduces a method that uses endoscopists' gaze patterns to improve the accuracy of classifying colorectal polyps using AI during endoscopy.
Contribution
The novel use of gaze-based attention as an auxiliary signal to train CNNs for colorectal polyp classification.
Findings
EfficientNet_b1 with gaze supervision achieved 86.96% test accuracy, outperforming the model without gaze data.
The model's class activation maps showed improved alignment with endoscopists' attention patterns.
The method demonstrated higher precision, recall, F1 score, and AUC compared to the baseline model.
Abstract
Colorectal polyps are potential precursor lesions of colorectal cancer. Accurate classification of colorectal polyps during endoscopy is crucial for early diagnosis and effective treatment. Automatic and accurate classification of colorectal polyps based on convolutional neural networks (CNNs) during endoscopy is vital for assisting endoscopists in diagnosis and treatment. However, this task remains challenging due to difficulties in the data acquisition and annotation processes, the poor interpretability of the data output, and the lack of widespread acceptance of the CNN models by clinicians. This study proposes an innovative approach that utilizes gaze attention information from endoscopists as an auxiliary supervisory signal to train a CNN-based model for the classification of colorectal polyps. Gaze information from the reading of endoscopic images was first recorded through an…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20
Figure 21
Figure 22
Figure 23
Figure 24Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRadiomics and Machine Learning in Medical Imaging · Image Retrieval and Classification Techniques · Multimodal Machine Learning Applications