The structure of broad topics on the Web
Soumen Chakrabarti, Mukul M. Joshi, Kunal Punera, David M. Pennock

TL;DR
This paper explores the structure of Web content communities by integrating topic taxonomies with graph analysis, revealing insights into content distribution, link patterns, and the effectiveness of random walk algorithms for sampling and ranking.
Contribution
It introduces a method combining topic taxonomies with Web graph analysis to understand content-based communities and their properties, enhancing community detection and ranking strategies.
Findings
Web pages about the same broad topic tend to cluster together.
Random walk algorithms can effectively sample topic distributions on the Web.
Topic mixing distance influences the effectiveness of global PageRank.
Abstract
The Web graph is a giant social network whose properties have been measured and modeled extensively in recent years. Most such studies concentrate on the graph structure alone, and do not consider textual properties of the nodes. Consequently, Web communities have been characterized purely in terms of graph structure and not on page content. We propose that a topic taxonomy such as Yahoo! or the Open Directory provides a useful framework for understanding the structure of content-based clusters and communities. In particular, using a topic taxonomy and an automatic classifier, we can measure the background distribution of broad topics on the Web, and analyze the capability of recent random walk algorithms to draw samples which follow such distributions. In addition, we can measure the probability that a page about one broad topic will link to another broad topic. Extending this…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComplex Network Analysis Techniques · Web Data Mining and Analysis · Peer-to-Peer Network Technologies
