Recognizing Image Objects by Relational Analysis Using Heterogeneous Superpixels and Deep Convolutional Features

TL;DR

This paper introduces a novel deep neural network architecture that leverages superpixels and capsule features to improve image object recognition, offering enhanced interpretability and structured analysis.

Contribution

The paper presents a new neural architecture integrating superpixels with capsule networks for structured image analysis and improved interpretability.

Findings

Superpixel-based features can be effectively learned using capsules.

Structured representation improves interpretability of image classification.

The proposed model outperforms baseline deep neural networks in accuracy.

Abstract

Superpixel-based methodologies have become increasingly popular in computer vision, especially when the computation is too expensive in time or memory to perform with a large number of pixels or features. However, rarely is superpixel segmentation examined within the context of deep convolutional neural network architectures. This paper presents a novel neural architecture that exploits the superpixel feature space. The visual feature space is organized using superpixels to provide the neural network with a substructure of the images. As the superpixels associate the visual feature space with parts of the objects in an image, the visual feature space is transformed into a structured vector representation per superpixel. It is shown that it is feasible to learn superpixel features using capsules and it is potentially beneficial to perform image analysis in such a structured manner. This novel deep learning architecture is examined in the context of an image classification task, highlighting explicit interpretability (explainability) of the network's decision making. The results are compared against a baseline deep neural model, as well as among superpixel capsule networks with a variety of hyperparameter settings.