"Forgetting" in Machine Learning and Beyond: A Survey

TL;DR

This survey explores the concept of forgetting in machine learning, highlighting its benefits, applications, challenges, and ethical considerations, inspired by neuroscientific insights into forgetting as an adaptive and beneficial process.

Contribution

It provides a comprehensive overview of forgetting mechanisms in machine learning, emphasizing their potential to improve performance and privacy, and discusses future research directions.

Findings

Forgetting can enhance model generalization and prevent overfitting.

Incorporating forgetting mechanisms can improve data privacy.

Challenges include designing effective forgetting algorithms and addressing ethical issues.

Abstract

This survey investigates the multifaceted nature of forgetting in machine learning, drawing insights from neuroscientific research that posits forgetting as an adaptive function rather than a defect, enhancing the learning process and preventing overfitting. This survey focuses on the benefits of forgetting and its applications across various machine learning sub-fields that can help improve model performance and enhance data privacy. Moreover, the paper discusses current challenges, future directions, and ethical considerations regarding the integration of forgetting mechanisms into machine learning models.