Friendships, Rivalries, and Trysts: Characterizing Relations between Ideas in Texts

TL;DR

This paper introduces a novel framework to analyze relationships between ideas in texts by examining their co-occurrence and prevalence over time, revealing complex interactions like cooperation and rivalry.

Contribution

It is the first systematic method to characterize idea relations based on text data, independent of idea representation, using co-occurrence and prevalence correlation metrics.

Findings

Ideas can be correlated in prevalence but rarely co-occur, indicating complex relationships.

The approach uncovers diverse idea interactions such as cooperation and rivalry.

Case studies demonstrate the framework's ability to reveal nuanced idea relations.

Abstract

Understanding how ideas relate to each other is a fundamental question in many domains, ranging from intellectual history to public communication. Because ideas are naturally embedded in texts, we propose the first framework to systematically characterize the relations between ideas based on their occurrence in a corpus of documents, independent of how these ideas are represented. Combining two statistics --- cooccurrence within documents and prevalence correlation over time --- our approach reveals a number of different ways in which ideas can cooperate and compete. For instance, two ideas can closely track each other's prevalence over time, and yet rarely cooccur, almost like a "cold war" scenario. We observe that pairwise cooccurrence and prevalence correlation exhibit different distributions. We further demonstrate that our approach is able to uncover intriguing relations between ideas through in-depth case studies on news articles and research papers.