Fine-Pruning: A Biologically Inspired Algorithm for Personalization of Machine Learning Models

TL;DR

This paper introduces Fine-Pruning, a biologically inspired algorithm that personalizes neural networks by mimicking brain pruning, achieving high sparsity and accuracy without backpropagation or labeled data.

Contribution

The work presents a novel pruning-based training method inspired by biological learning, reducing computational resources and eliminating the need for labeled datasets.

Findings

Achieved 70% sparsity in models

Improved accuracy to around 90%

Enabled personalization of speech and image models

Abstract

Neural networks have long strived to emulate the learning capabilities of the human brain. While deep neural networks (DNNs) draw inspiration from the brain in neuron design, their training methods diverge from biological foundations. Backpropagation, the primary training method for DNNs, requires substantial computational resources and fully labeled datasets, presenting major bottlenecks in development and application. This work demonstrates that by returning to biomimicry, specifically mimicking how the brain learns through pruning, we can solve various classical machine learning problems while utilizing orders of magnitude fewer computational resources and no labels. Our experiments successfully personalized multiple speech recognition and image classification models, including ResNet50 on ImageNet, resulting in increased sparsity of approximately 70\% while simultaneously improving model accuracy to around 90\%, all without the limitations of backpropagation. This biologically inspired approach offers a promising avenue for efficient, personalized machine learning models in resource-constrained environments.