Data Uncertainty without Prediction Models

TL;DR

This paper introduces a novel uncertainty estimation method called Distance-weighted Class Impurity that does not rely on prediction models, enabling effective active learning with less data.

Contribution

The paper proposes a new uncertainty estimation technique based on distances and class impurities, independent of prediction models, improving active learning efficiency.

Findings

Effective uncertainty estimation without prediction models

Comparable or superior performance in active learning tasks

Versatile applicability across different datasets

Abstract

Data acquisition processes for machine learning are often costly. To construct a high-performance prediction model with fewer data, a degree of difficulty in prediction is often deployed as the acquisition function in adding a new data point. The degree of difficulty is referred to as uncertainty in prediction models. We propose an uncertainty estimation method named a Distance-weighted Class Impurity without explicit use of prediction models. We estimated uncertainty using distances and class impurities around the location, and compared it with several methods based on prediction models for uncertainty estimation by active learning tasks. We verified that the Distance-weighted Class Impurity works effectively regardless of prediction models.