PDR-CapsNet: an Energy-Efficient Parallel Approach to Dynamic Routing in Capsule Networks

TL;DR

PDR-CapsNet is a novel, energy-efficient parallel dynamic routing approach for Capsule Networks that improves accuracy, reduces resource consumption, and speeds up inference on image classification tasks.

Contribution

It introduces a parallelization strategy for CapsNets, significantly enhancing efficiency and performance over traditional CapsNet architectures.

Findings

Achieves 83.55% accuracy on CIFAR-10.

Uses 87.26% fewer parameters than CapsNet.

Runs 3x faster with 7.29J less energy on GPU.

Abstract

Convolutional Neural Networks (CNNs) have produced state-of-the-art results for image classification tasks. However, they are limited in their ability to handle rotational and viewpoint variations due to information loss in max-pooling layers. Capsule Networks (CapsNets) employ a computationally-expensive iterative process referred to as dynamic routing to address these issues. CapsNets, however, often fall short on complex datasets and require more computational resources than CNNs. To overcome these challenges, we introduce the Parallel Dynamic Routing CapsNet (PDR-CapsNet), a deeper and more energy-efficient alternative to CapsNet that offers superior performance, less energy consumption, and lower overfitting rates. By leveraging a parallelization strategy, PDR-CapsNet mitigates the computational complexity of CapsNet and increases throughput, efficiently using hardware resources. As a result, we achieve 83.55\% accuracy while requiring 87.26\% fewer parameters, 32.27\% and 47.40\% fewer MACs, and Flops, achieving 3x faster inference and 7.29J less energy consumption on a 2080Ti GPU with 11GB VRAM compared to CapsNet and for the CIFAR-10 dataset.