# Weight Friction: A Simple Method to Overcome Catastrophic Forgetting and Enable Continual Learning

## TL;DR

This paper introduces weight friction, a simple and efficient method inspired by neurology and physics, to prevent catastrophic forgetting in neural networks and facilitate continual learning across multiple tasks.

## Contribution

The paper proposes weight friction, a novel modification to gradient descent, enabling neural networks to learn sequential tasks without forgetting, with improved efficiency.

## Key findings

- Performs comparably to existing methods in preventing forgetting.
- Operates efficiently with lower computational and memory costs.
- Converges at a rate similar to stochastic gradient descent.

## Abstract

In recent years, deep neural networks have found success in replicating human-level cognitive skills, yet they suffer from several major obstacles. One significant limitation is the inability to learn new tasks without forgetting previously learned tasks, a shortcoming known as catastrophic forgetting. In this research, we propose a simple method to overcome catastrophic forgetting and enable continual learning in neural networks. We draw inspiration from principles in neurology and physics to develop the concept of weight friction. Weight friction operates by a modification to the update rule in the gradient descent optimization method. It converges at a rate comparable to that of the stochastic gradient descent algorithm and can operate over multiple task domains. It performs comparably to current methods while offering improvements in computation and memory efficiency.