Hierarchical learning of grids of microtopics

TL;DR

This paper introduces a hierarchical counting grid model that improves microtopic extraction, classification accuracy, and avoids local minima, demonstrating its effectiveness in embedding images and analyzing microtopics.

Contribution

It extends the counting grid model with hierarchical reasoning, enhancing microtopic extraction and classification, especially from small datasets.

Findings

Hierarchical models improve microtopic coherence and diversity.

The approach enhances classification accuracy over basic models.

Effective for embedding raw images and extracting microtopics.

Abstract

The counting grid is a grid of microtopics, sparse word/feature distributions. The generative model associated with the grid does not use these microtopics individually. Rather, it groups them in overlapping rectangular windows and uses these grouped microtopics as either mixture or admixture components. This paper builds upon the basic counting grid model and it shows that hierarchical reasoning helps avoid bad local minima, produces better classification accuracy and, most interestingly, allows for extraction of large numbers of coherent microtopics even from small datasets. We evaluate this in terms of consistency, diversity and clarity of the indexed content, as well as in a user study on word intrusion tasks. We demonstrate that these models work well as a technique for embedding raw images and discuss interesting parallels between hierarchical CG models and other deep architectures.