Multi-Label Learning with Deep Forest

TL;DR

This paper introduces Multi-Label Deep Forest (MLDF), a novel ensemble method that leverages label correlations and adaptively adjusts model complexity to improve multi-label learning performance without relying on deep neural networks.

Contribution

The paper proposes MLDF with measure-aware feature reuse and layer growth mechanisms, addressing overfitting and multiple evaluation measures in multi-label learning.

Findings

MLDF outperforms existing methods on six evaluation measures.

MLDF effectively discovers label correlations.

MLDF reduces overfitting through controlled model complexity.

Abstract

In multi-label learning, each instance is associated with multiple labels and the crucial task is how to leverage label correlations in building models. Deep neural network methods usually jointly embed the feature and label information into a latent space to exploit label correlations. However, the success of these methods highly depends on the precise choice of model depth. Deep forest is a recent deep learning framework based on tree model ensembles, which does not rely on backpropagation. We consider the advantages of deep forest models are very appropriate for solving multi-label problems. Therefore we design the Multi-Label Deep Forest (MLDF) method with two mechanisms: measure-aware feature reuse and measure-aware layer growth. The measure-aware feature reuse mechanism reuses the good representation in the previous layer guided by confidence. The measure-aware layer growth mechanism ensures MLDF gradually increase the model complexity by performance measure. MLDF handles two challenging problems at the same time: one is restricting the model complexity to ease the overfitting issue; another is optimizing the performance measure on user's demand since there are many different measures in the multi-label evaluation. Experiments show that our proposal not only beats the compared methods over six measures on benchmark datasets but also enjoys label correlation discovery and other desired properties in multi-label learning.