The Identification of Context-Sensitive Features: A Formal Definition of Context for Concept Learning

TL;DR

This paper provides a formal framework to identify context-sensitive features in machine learning, clarifying their impact on learning performance and enabling automated detection of such features.

Contribution

It introduces formal definitions for primary, contextual, and irrelevant features, and defines what it means for a feature to be context-sensitive, correcting previous flaws.

Findings

Formal definitions distinguish feature types and context sensitivity.

Automated identification of context-sensitive features is possible.

Clarifies the relationship between context sensitivity and feature relevance.

Abstract

A large body of research in machine learning is concerned with supervised learning from examples. The examples are typically represented as vectors in a multi-dimensional feature space (also known as attribute-value descriptions). A teacher partitions a set of training examples into a finite number of classes. The task of the learning algorithm is to induce a concept from the training examples. In this paper, we formally distinguish three types of features: primary, contextual, and irrelevant features. We also formally define what it means for one feature to be context-sensitive to another feature. Context-sensitive features complicate the task of the learner and potentially impair the learner's performance. Our formal definitions make it possible for a learner to automatically identify context-sensitive features. After context-sensitive features have been identified, there are several strategies that the learner can employ for managing the features; however, a discussion of these strategies is outside of the scope of this paper. The formal definitions presented here correct a flaw in previously proposed definitions. We discuss the relationship between our work and a formal definition of relevance.