How to Do Machine Learning with Small Data? -- A Review from an Industrial Perspective

TL;DR

This review paper discusses the challenges, definitions, and industrial applications of machine learning with small data, emphasizing its importance when data is limited and exploring various approaches to address these issues.

Contribution

It provides a comprehensive overview of small data in industrial machine learning, including challenges, definitions, and a taxonomy of approaches tailored for limited data scenarios.

Findings

Identifies five critical challenges in small data machine learning: unlabeled, imbalanced, missing, insufficient data, and rare events.

Defines small data in contrast to big data with specific characteristics.

Provides a taxonomy of machine learning approaches suitable for small data in industrial contexts.

Abstract

Artificial intelligence experienced a technological breakthrough in science, industry, and everyday life in the recent few decades. The advancements can be credited to the ever-increasing availability and miniaturization of computational resources that resulted in exponential data growth. However, because of the insufficient amount of data in some cases, employing machine learning in solving complex tasks is not straightforward or even possible. As a result, machine learning with small data experiences rising importance in data science and application in several fields. The authors focus on interpreting the general term of "small data" and their engineering and industrial application role. They give a brief overview of the most important industrial applications of machine learning and small data. Small data is defined in terms of various characteristics compared to big data, and a machine learning formalism was introduced. Five critical challenges of machine learning with small data in industrial applications are presented: unlabeled data, imbalanced data, missing data, insufficient data, and rare events. Based on those definitions, an overview of the considerations in domain representation and data acquisition is given along with a taxonomy of machine learning approaches in the context of small data.