Application of R\'enyi and Tsallis Entropies to Topic Modeling Optimization

TL;DR

This paper explores using Rényi and Tsallis entropies to determine the optimal number of topics in large text collections, emphasizing semantic stability's role through thermodynamics-based entropy calculations.

Contribution

It introduces a novel approach employing Rényi and Tsallis entropies for identifying the optimal number of topics in topic modeling.

Findings

Entropy measures correlate with semantic stability.

Thermodynamics-based entropy calculations are effective.

Optimal topic number can be identified using entropy analysis.

Abstract

This is full length article (draft version) where problem number of topics in Topic Modeling is discussed. We proposed idea that Renyi and Tsallis entropy can be used for identification of optimal number in large textual collections. We also report results of numerical experiments of Semantic stability for 4 topic models, which shows that semantic stability play very important role in problem topic number. The calculation of Renyi and Tsallis entropy based on thermodynamics approach.