Hidden Tree Markov Networks: Deep and Wide Learning for Structured Data

TL;DR

The paper proposes Hidden Tree Markov Networks, a hybrid deep and wide neural architecture combining generative models for trees with neural discriminative layers, improving structured data classification.

Contribution

It introduces a modular, hybrid model that fuses generative tree models with neural networks, enabling deep and wide learning for structured data.

Findings

Outperforms state-of-the-art syntactic kernels

Outperforms generative kernels based on the same probabilistic model

Demonstrates effectiveness on structured data tasks

Abstract

The paper introduces the Hidden Tree Markov Network (HTN), a neuro-probabilistic hybrid fusing the representation power of generative models for trees with the incremental and discriminative learning capabilities of neural networks. We put forward a modular architecture in which multiple generative models of limited complexity are trained to learn structural feature detectors whose outputs are then combined and integrated by neural layers at a later stage. In this respect, the model is both deep, thanks to the unfolding of the generative models on the input structures, as well as wide, given the potentially large number of generative modules that can be trained in parallel. Experimental results show that the proposed approach can outperform state-of-the-art syntactic kernels as well as generative kernels built on the same probabilistic model as the HTN.