A method of supervised learning from conflicting data with hidden contexts

TL;DR

This paper introduces LEAF, a supervised learning method that manages conflicting data from hidden contexts by learning to allocate data to different models, supported by theoretical analysis and empirical validation.

Contribution

The paper proposes LEAF, a novel approach that handles conflicting data from unobservable domains by learning data allocations, extending supervised learning to more complex, context-dependent scenarios.

Findings

LEAF effectively manages conflicting data in synthetic and real-world tasks.

Theoretical analysis confirms the validity of the allocation mechanism.

Empirical results demonstrate improved performance over standard methods.

Abstract

Conventional supervised learning assumes a stable input-output relationship. However, this assumption fails in open-ended training settings where the input-output relationship depends on hidden contexts. In this work, we formulate a more general supervised learning problem in which training data is drawn from multiple unobservable domains, each potentially exhibiting distinct input-output maps. This inherent conflict in data renders standard empirical risk minimization training ineffective. To address this challenge, we propose a method LEAF that introduces an allocation function, which learns to assign conflicting data to different predictive models. We establish a connection between LEAF and a variant of the Expectation-Maximization algorithm, allowing us to derive an analytical expression for the allocation function. Finally, we provide a theoretical analysis of LEAF and empirically validate its effectiveness on both synthetic and real-world tasks involving conflicting data.