Jointly Modeling Topics and Intents with Global Order Structure

TL;DR

This paper introduces GMM-LDA, a Bayesian model that captures document structure by jointly modeling topics and rhetorical intents, incorporating order information and enabling supervised learning.

Contribution

It presents a novel unsupervised and supervised Bayesian model that integrates topic and intent modeling with global order structure in documents.

Findings

Outperforms state-of-the-art baselines in experiments

Effectively models document intent and topic structure

Flexible to incorporate annotations and supervised learning

Abstract

Modeling document structure is of great importance for discourse analysis and related applications. The goal of this research is to capture the document intent structure by modeling documents as a mixture of topic words and rhetorical words. While the topics are relatively unchanged through one document, the rhetorical functions of sentences usually change following certain orders in discourse. We propose GMM-LDA, a topic modeling based Bayesian unsupervised model, to analyze the document intent structure cooperated with order information. Our model is flexible that has the ability to combine the annotations and do supervised learning. Additionally, entropic regularization can be introduced to model the significant divergence between topics and intents. We perform experiments in both unsupervised and supervised settings, results show the superiority of our model over several state-of-the-art baselines.