Neural Message Passing for Multi-Label Classification

TL;DR

This paper introduces Label Message Passing (LaMP), a neural network approach that models label interactions in multi-label classification using attention-based message passing, leading to improved accuracy and interpretability.

Contribution

LaMP is a novel, simple, and highly parallelizable neural network model that explicitly models label interactions via attention-based message passing for multi-label classification.

Findings

Outperforms state-of-the-art on seven real-world datasets

Provides interpretable insights into label dependencies

Efficiently handles dense labels with high accuracy

Abstract

Multi-label classification (MLC) is the task of assigning a set of target labels for a given sample. Modeling the combinatorial label interactions in MLC has been a long-haul challenge. We propose Label Message Passing (LaMP) Neural Networks to efficiently model the joint prediction of multiple labels. LaMP treats labels as nodes on a label-interaction graph and computes the hidden representation of each label node conditioned on the input using attention-based neural message passing. Attention enables LaMP to assign different importance to neighbor nodes per label, learning how labels interact (implicitly). The proposed models are simple, accurate, interpretable, structure-agnostic, and applicable for predicting dense labels since LaMP is incredibly parallelizable. We validate the benefits of LaMP on seven real-world MLC datasets, covering a broad spectrum of input/output types and outperforming the state-of-the-art results. Notably, LaMP enables intuitive interpretation of how classifying each label depends on the elements of a sample and at the same time rely on its interaction with other labels. We provide our code and datasets at https://github.com/QData/LaMP