LLMs Between the Nodes: Community Discovery Beyond Vectors

TL;DR

This paper explores how Large Language Models can be integrated with graph structure to improve community detection in social networks, demonstrating promising results especially with instruction-tuned models and prompt engineering.

Contribution

It introduces the CommLLM framework that combines GPT-4o with graph-aware prompts for community detection, advancing LLM applications in graph analysis.

Findings

LLMs can effectively identify communities in small to medium graphs.

Graph-aware prompting enhances community detection accuracy.

Instruction-tuned models improve coherence of detected communities.

Abstract

Community detection in social network graphs plays a vital role in uncovering group dynamics, influence pathways, and the spread of information. Traditional methods focus primarily on graph structural properties, but recent advancements in Large Language Models (LLMs) open up new avenues for integrating semantic and contextual information into this task. In this paper, we present a detailed investigation into how various LLM-based approaches perform in identifying communities within social graphs. We introduce a two-step framework called CommLLM, which leverages the GPT-4o model along with prompt-based reasoning to fuse language model outputs with graph structure. Evaluations are conducted on six real-world social network datasets, measuring performance using key metrics such as Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), Variation of Information (VOI), and cluster purity. Our findings reveal that LLMs, particularly when guided by graph-aware strategies, can be successfully applied to community detection tasks in small to medium-sized graphs. We observe that the integration of instruction-tuned models and carefully engineered prompts significantly improves the accuracy and coherence of detected communities. These insights not only highlight the potential of LLMs in graph-based research but also underscore the importance of tailoring model interactions to the specific structure of graph data.