An Improvement for Capsule Networks using Depthwise Separable Convolution

TL;DR

This paper enhances Capsule Networks by integrating Depthwise Separable Convolution, reducing parameters, increasing stability, and achieving competitive accuracy, especially on larger images, while also comparing with state-of-the-art transfer learning models.

Contribution

First integration of Depthwise Separable Convolution into Capsule Networks, improving efficiency and stability while maintaining competitive accuracy.

Findings

Reduced model parameters significantly.

Achieved better performance on larger images.

Capsule Networks perform comparably to deep learning models.

Abstract

Capsule Networks face a critical problem in computer vision in the sense that the image background can challenge its performance, although they learn very well on training data. In this work, we propose to improve Capsule Networks' architecture by replacing the Standard Convolution with a Depthwise Separable Convolution. This new design significantly reduces the model's total parameters while increases stability and offers competitive accuracy. In addition, the proposed model on $64\times64$ pixel images outperforms standard models on $32\times32$ and $64\times64$ pixel images. Moreover, we empirically evaluate these models with Deep Learning architectures using state-of-the-art Transfer Learning networks such as Inception V3 and MobileNet V1. The results show that Capsule Networks can perform comparably against Deep Learning models. To the best of our knowledge, we believe that this is the first work on the integration of Depthwise Separable Convolution into Capsule Networks.