Exploring the Performance of Perforated Backpropagation through Further Experiments

TL;DR

This paper investigates Perforated Backpropagation, a neural network optimization inspired by biological neurons, demonstrating significant model compression and accuracy improvements through collaborative experiments.

Contribution

It provides empirical evidence of Perforated Backpropagation's effectiveness in real-world ML projects, expanding understanding of its practical benefits.

Findings

Up to 90% model compression without accuracy loss

Up to 16% accuracy increase in tested models

Validated through experiments at a ML hackathon

Abstract

Perforated Backpropagation is a neural network optimization technique based on modern understanding of the computational importance of dendrites within biological neurons. This paper explores further experiments from the original publication, generated from a hackathon held at the Carnegie Mellon Swartz Center in February 2025. Students and local Pittsburgh ML practitioners were brought together to experiment with the Perforated Backpropagation algorithm on the datasets and models which they were using for their projects. Results showed that the system could enhance their projects, with up to 90% model compression without negative impact on accuracy, or up to 16% increased accuracy of their original models.