Discovering and Exploiting Entailment Relationships in Multi-Label Learning

TL;DR

This paper introduces a probabilistic method that enforces label relationships in multi-label learning, improving precision by discovering and exploiting entailment and exclusion relationships among labels.

Contribution

It presents a novel Bayesian network-based approach to automatically discover and incorporate deterministic label relationships into multi-label models.

Findings

Significant improvements in mean average precision across 12 datasets.

Effective discovery of positive entailment and exclusion relationships.

Robustness of the method in enforcing label dependencies.

Abstract

This work presents a sound probabilistic method for enforcing adherence of the marginal probabilities of a multi-label model to automatically discovered deterministic relationships among labels. In particular we focus on discovering two kinds of relationships among the labels. The first one concerns pairwise positive entailement: pairs of labels, where the presence of one implies the presence of the other in all instances of a dataset. The second concerns exclusion: sets of labels that do not coexist in the same instances of the dataset. These relationships are represented with a Bayesian network. Marginal probabilities are entered as soft evidence in the network and adjusted through probabilistic inference. Our approach offers robust improvements in mean average precision compared to the standard binary relavance approach across all 12 datasets involved in our experiments. The discovery process helps interesting implicit knowledge to emerge, which could be useful in itself.