Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations

TL;DR

This paper proposes approximate implementations of softmax and squash operations to enable efficient deployment of Capsule Networks on edge devices, balancing accuracy with hardware resource savings.

Contribution

It introduces approximate variants of key CapsNet operations and evaluates their hardware efficiency and impact on model accuracy.

Findings

Significant reductions in area and power consumption.

Maintained acceptable accuracy levels with approximate functions.

Tradeoffs between hardware efficiency and model performance are characterized.

Abstract

Complex Deep Neural Networks such as Capsule Networks (CapsNets) exhibit high learning capabilities at the cost of compute-intensive operations. To enable their deployment on edge devices, we propose to leverage approximate computing for designing approximate variants of the complex operations like softmax and squash. In our experiments, we evaluate tradeoffs between area, power consumption, and critical path delay of the designs implemented with the ASIC design flow, and the accuracy of the quantized CapsNets, compared to the exact functions.