Document Retrieval for Large Scale Content Analysis using Contextualized Dictionaries
Gregor Wiedemann, Andreas Niekler

TL;DR
This paper introduces a method for retrieving relevant documents in large collections for social science content analysis by using large, manually compiled dictionaries derived from topic models and co-occurrence data, improving retrieval accuracy.
Contribution
It proposes a novel document retrieval approach that leverages paradigmatic reference collections and large dictionaries, enhancing content analysis in social sciences.
Findings
Improved retrieval results over alternative key term extraction methods
Utilization of co-occurrence data enhances retrieval accuracy
Dictionary-based retrieval outperforms classic methods in content analysis tasks
Abstract
This paper presents a procedure to retrieve subsets of relevant documents from large text collections for Content Analysis, e.g. in social sciences. Document retrieval for this purpose needs to take account of the fact that analysts often cannot describe their research objective with a small set of key terms, especially when dealing with theoretical or rather abstract research interests. Instead, it is much easier to define a set of paradigmatic documents which reflect topics of interest as well as targeted manner of speech. Thus, in contrast to classic information retrieval tasks we employ manually compiled collections of reference documents to compose large queries of several hundred key terms, called dictionaries. We extract dictionaries via Topic Models and also use co-occurrence data from reference collections. Evaluations show that the procedure improves retrieval results for this…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Text Analysis Techniques · Topic Modeling · Semantic Web and Ontologies
