# An Automatic Approach for Document-level Topic Model Evaluation

## TL;DR

This paper highlights the limitations of topic-level evaluation for topic models and introduces an automatic document-level evaluation method that provides a more accurate assessment of model quality.

## Contribution

It presents a novel automatic evaluation approach focusing on document-level topic allocations, addressing discrepancies with traditional topic-level metrics.

## Key findings

- Topic-level and document-level evaluations can significantly differ.
- The proposed method reliably predicts model quality based on document data.
- Document-level analysis offers a more comprehensive view of topic model performance.

## Abstract

Topic models jointly learn topics and document-level topic distribution. Extrinsic evaluation of topic models tends to focus exclusively on topic-level evaluation, e.g. by assessing the coherence of topics. We demonstrate that there can be large discrepancies between topic- and document-level model quality, and that basing model evaluation on topic-level analysis can be highly misleading. We propose a method for automatically predicting topic model quality based on analysis of document-level topic allocations, and provide empirical evidence for its robustness.