DECAPS: Detail-Oriented Capsule Networks

TL;DR

DECAPS introduces a novel capsule network architecture with innovative routing, attention, and distillation techniques that significantly improve classification accuracy and localization in medical imaging datasets.

Contribution

The paper proposes DECAPS, a capsule network with Inverted Dynamic Routing, Peekaboo training, and distillation, achieving state-of-the-art results on large-scale medical datasets.

Findings

Achieves 92.82% AUC on CheXpert dataset.

Improves pneumonia localization precision from 41.7% to 80%.

Outperforms existing methods in accuracy and robustness.

Abstract

Capsule Networks (CapsNets) have demonstrated to be a promising alternative to Convolutional Neural Networks (CNNs). However, they often fall short of state-of-the-art accuracies on large-scale high-dimensional datasets. We propose a Detail-Oriented Capsule Network (DECAPS) that combines the strength of CapsNets with several novel techniques to boost its classification accuracies. First, DECAPS uses an Inverted Dynamic Routing (IDR) mechanism to group lower-level capsules into heads before sending them to higher-level capsules. This strategy enables capsules to selectively attend to small but informative details within the data which may be lost during pooling operations in CNNs. Second, DECAPS employs a Peekaboo training procedure, which encourages the network to focus on fine-grained information through a second-level attention scheme. Finally, the distillation process improves the robustness of DECAPS by averaging over the original and attended image region predictions. We provide extensive experiments on the CheXpert and RSNA Pneumonia datasets to validate the effectiveness of DECAPS. Our networks achieve state-of-the-art accuracies not only in classification (increasing the average area under ROC curves from 87.24% to 92.82% on the CheXpert dataset) but also in the weakly-supervised localization of diseased areas (increasing average precision from 41.7% to 80% for the RSNA Pneumonia detection dataset).