# Topically Driven Neural Language Model

## TL;DR

This paper introduces a neural language model that integrates document-level context via a topic model-like structure, improving language modeling and topic coherence compared to traditional sentence-based models.

## Contribution

It presents a novel neural architecture that combines language modeling with document context and topic coherence, enhancing interpretability and performance.

## Key findings

- Outperforms sentence-based models in perplexity
- Produces more coherent topics than standard LDA
- Can generate related sentences for topics

## Abstract

Language models are typically applied at the sentence level, without access to the broader document context. We present a neural language model that incorporates document context in the form of a topic model-like architecture, thus providing a succinct representation of the broader document context outside of the current sentence. Experiments over a range of datasets demonstrate that our model outperforms a pure sentence-based model in terms of language model perplexity, and leads to topics that are potentially more coherent than those produced by a standard LDA topic model. Our model also has the ability to generate related sentences for a topic, providing another way to interpret topics.