Cooperative Hierarchical Dirichlet Processes: Superposition vs. Maximization

TL;DR

This paper introduces the cooperative hierarchical Dirichlet process (CHDP), a Bayesian nonparametric model that captures complex many-to-many relationships in hierarchical data without fixing the number of hidden factors.

Contribution

It proposes a novel CHDP model with superposition and maximization measures, along with two constructive inference methods, enabling flexible modeling of cooperative hierarchical structures.

Findings

CHDP effectively models cooperative hierarchies in synthetic and real data.

The model demonstrates superior flexibility over fixed-topic models.

Experimental results validate CHDP's ability to capture complex relationships.

Abstract

The cooperative hierarchical structure is a common and significant data structure observed in, or adopted by, many research areas, such as: text mining (author-paper-word) and multi-label classification (label-instance-feature). Renowned Bayesian approaches for cooperative hierarchical structure modeling are mostly based on topic models. However, these approaches suffer from a serious issue in that the number of hidden topics/factors needs to be fixed in advance and an inappropriate number may lead to overfitting or underfitting. One elegant way to resolve this issue is Bayesian nonparametric learning, but existing work in this area still cannot be applied to cooperative hierarchical structure modeling. In this paper, we propose a cooperative hierarchical Dirichlet process (CHDP) to fill this gap. Each node in a cooperative hierarchical structure is assigned a Dirichlet process to model its weights on the infinite hidden factors/topics. Together with measure inheritance from hierarchical Dirichlet process, two kinds of measure cooperation, i.e., superposition and maximization, are defined to capture the many-to-many relationships in the cooperative hierarchical structure. Furthermore, two constructive representations for CHDP, i.e., stick-breaking and international restaurant process, are designed to facilitate the model inference. Experiments on synthetic and real-world data with cooperative hierarchical structures demonstrate the properties and the ability of CHDP for cooperative hierarchical structure modeling and its potential for practical application scenarios.