Perforated Neural Networks for Keyword Spotting

TL;DR

This paper introduces Perforated Backpropagation with Dendrite Nodes, significantly improving keyword spotting accuracy and efficiency on edge devices by outperforming traditional models at lower parameter counts.

Contribution

It demonstrates that dendritic models trained with Perforated Backpropagation outperform traditional architectures in accuracy and size for edge keyword spotting tasks.

Findings

Dendritic models achieved 0.933 accuracy with only 1,500 parameters.

Compared to baseline, the best model used fewer parameters (1,500 vs. 4,000).

Perforated Backpropagation enhances both accuracy and model efficiency.

Abstract

Edge machine learning presents a unique set of constraints not encountered in cloud-scale model deployment: strict memory budgets, limited compute, and non-negotiable accuracy thresholds must all be satisfied simultaneously. Existing compression and optimization techniques can trade one resource for another, but rarely improve both accuracy and model size at the same time. This paper presents the application of Perforated Backpropagation to keyword spotting on the Edge Impulse platform, an experiment that won the Best Model award at the Edge Impulse 2025 Hackathon in December 2025. By adding artificial Dendrite Nodes to a standard convolutional neural network trained on the Edge Impulse keyword spotting tutorial pipeline, we demonstrate that dendritic models outperform traditional architectures at every level of parameter count and at every accuracy threshold tested across 800 hyperparameter trials. The best dendritic model achieved a test accuracy of 0.933 with only 1,500 parameters, versus the baseline accuracy of 0.921 requiring approximately 4,000 parameters. These results suggest that Perforated Backpropagation is a powerful addition to the edge AI engineer's toolkit, offering simultaneous gains in both model quality and deployment efficiency.