Information-theoretic Limits for Testing Community Structures in Weighted Networks

TL;DR

This paper establishes the fundamental limits for testing community structures in weighted networks, quantifies information loss from dichotomization, and introduces new effective tests with practical applications.

Contribution

It derives the information-theoretic limits for community detection in weighted networks and proposes new tests, addressing a gap in hypothesis testing for weighted graphs.

Findings

Identified the sharp information-theoretic limit for community detection.

Quantified the information loss from dichotomizing weighted networks.

Proposed new test statistics with good empirical performance.

Abstract

Community detection refers to the problem of clustering the nodes of a network into groups. Existing inferential methods for community structure mainly focus on unweighted (binary) networks. Many real-world networks are nonetheless weighted and a common practice is to dichotomize a weighted network to an unweighted one which is known to result in information loss. Literature on hypothesis testing in the latter situation is still missing. In this paper, we study the problem of testing the existence of community structure in weighted networks. Our contributions are threefold: (a). We use the (possibly infinite-dimensional) exponential family to model the weights and derive the sharp information-theoretic limit for the existence of consistent test. Within the limit, any test is inconsistent; and beyond the limit, we propose a useful consistent test. (b). Based on the information-theoretic limits, we provide the first formal way to quantify the loss of information incurred by dichotomizing weighted graphs into unweighted graphs in the context of hypothesis testing. (c). We propose several new and practically useful test statistics. Simulation study show that the proposed tests have good performance. Finally, we apply the proposed tests to an animal social network.