pRSL: Interpretable Multi-label Stacking by Learning Probabilistic Rules

TL;DR

pRSL introduces an interpretable probabilistic rule-based stacking method for multi-label classification, effectively modeling class relationships and achieving state-of-the-art results on benchmark datasets.

Contribution

It presents a novel probabilistic rule stacking approach using propositional logic and belief propagation, including a multicategorical noisy-or generalization.

Findings

Achieves state-of-the-art performance on benchmarks

Introduces a multicategorical noisy-or gate

Analyzes belief propagation in noisy-or networks

Abstract

A key task in multi-label classification is modeling the structure between the involved classes. Modeling this structure by probabilistic and interpretable means enables application in a broad variety of tasks such as zero-shot learning or learning from incomplete data. In this paper, we present the probabilistic rule stacking learner (pRSL) which uses probabilistic propositional logic rules and belief propagation to combine the predictions of several underlying classifiers. We derive algorithms for exact and approximate inference and learning, and show that pRSL reaches state-of-the-art performance on various benchmark datasets. In the process, we introduce a novel multicategorical generalization of the noisy-or gate. Additionally, we report simulation results on the quality of loopy belief propagation algorithms for approximate inference in bipartite noisy-or networks.