Ignorance-Aware Approaches and Algorithms for Prototype Selection in Machine Learning

TL;DR

This paper explores the novel idea of incorporating ignorance zones within data space into machine learning models, proposing algorithms for their discovery and using this insight to enhance prototype selection for improved classifier performance.

Contribution

It introduces ignorance-aware algorithms for discovering and visualizing ignorance zones, and proposes new prototype selection techniques that leverage ignorance to improve nearest neighbor classifiers.

Findings

Ignorance zones can be effectively visualized in 2D data spaces.

Ignorance-aware prototype selection improves classifier accuracy.

Algorithms are validated on artificial and real datasets.

Abstract

Operating with ignorance is an important concern of the Machine Learning research, especially when the objective is to discover knowledge from the imperfect data. Data mining (driven by appropriate knowledge discovery tools) is about processing available (observed, known and understood) samples of data aiming to build a model (e.g., a classifier) to handle data samples, which are not yet observed, known or understood. These tools traditionally take samples of the available data (known facts) as an input for learning. We want to challenge the indispensability of this approach and we suggest considering the things the other way around. What if the task would be as follows: how to learn a model based on our ignorance, i.e. by processing the shape of 'voids' within the available data space? Can we improve traditional classification by modeling also the ignorance? In this paper, we provide some algorithms for the discovery and visualizing of the ignorance zones in two-dimensional data spaces and design two ignorance-aware smart prototype selection techniques (incremental and adversarial) to improve the performance of the nearest neighbor classifiers. We present experiments with artificial and real datasets to test the concept of the usefulness of ignorance discovery in machine learning.