TraceCaps: A Capsule-based Neural Network for Semantic Segmentation

TL;DR

TraceCaps introduces a capsule-based neural network for semantic segmentation that leverages part-whole dependencies to improve accuracy and interpretability over traditional FCN models, demonstrated on MNIST and neuroimages.

Contribution

The paper presents a novel capsule-based segmentation model that explicitly models part-whole relationships and offers an end-to-end framework surpassing current FCN methods.

Findings

Significantly improved segmentation accuracy on MNIST and neuroimages.

Explicit modeling of part-whole dependencies enhances interpretability.

Outperforms state-of-the-art FCN variants in experiments.

Abstract

In this paper, we propose a capsule-based neural network model to solve the semantic segmentation problem. By taking advantage of the extractable part-whole dependencies available in capsule layers, we derive the probabilities of the class labels for individual capsules through a recursive, layer-by-layer procedure. We model this procedure as a traceback pipeline and take it as a central piece to build an end-to-end segmentation network. Under the proposed framework, image-level class labels and object boundaries are jointly sought in an explicit manner, which poses a significant advantage over the state-of-the-art fully convolutional network (FCN) solutions. With the capability to extracted part-whole information, our traceback pipeline can potentially be utilized as the building blocks to design interpretable neural networks. Experiments conducted on modified MNIST and neuroimages demonstrate that our model considerably enhance the segmentation performance compared to the leading FCN variants.