Deep tree-ensembles for multi-output prediction

TL;DR

This paper introduces a novel deep tree-ensemble model that enhances multi-output prediction tasks like multi-label classification and multi-target regression by leveraging tree-embeddings for improved feature representation.

Contribution

The paper proposes a new deep tree-ensemble architecture that incorporates tree-embeddings at each layer, specifically addressing multi-output prediction tasks.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Effective in multi-label classification and multi-target regression.

Provides superior results compared to existing approaches.

Abstract

Recently, deep neural networks have expanded the state-of-art in various scientific fields and provided solutions to long standing problems across multiple application domains. Nevertheless, they also suffer from weaknesses since their optimal performance depends on massive amounts of training data and the tuning of an extended number of parameters. As a countermeasure, some deep-forest methods have been recently proposed, as efficient and low-scale solutions. Despite that, these approaches simply employ label classification probabilities as induced features and primarily focus on traditional classification and regression tasks, leaving multi-output prediction under-explored. Moreover, recent work has demonstrated that tree-embeddings are highly representative, especially in structured output prediction. In this direction, we propose a novel deep tree-ensemble (DTE) model, where every layer enriches the original feature set with a representation learning component based on tree-embeddings. In this paper, we specifically focus on two structured output prediction tasks, namely multi-label classification and multi-target regression. We conducted experiments using multiple benchmark datasets and the obtained results confirm that our method provides superior results to state-of-the-art methods in both tasks.