T-Stochastic Graphs

TL;DR

This paper introduces T-Stochastic Graphs, a flexible probabilistic model for hierarchical clustering that addresses non-ultrametric structures in social networks, combining spectral methods with phylogenetic algorithms and proving their statistical consistency.

Contribution

It proposes the T-Stochastic Graph model, a general framework for hierarchical clustering without ultrametric constraints, and develops a spectral clustering approach with proven consistency.

Findings

Empirical networks often violate ultrametric assumptions.

The spectral clustering method is statistically consistent.

The T-Stochastic Graph model generalizes existing hierarchical models.

Abstract

Previous statistical approaches to hierarchical clustering for social network analysis all construct an "ultrametric" hierarchy. While the assumption of ultrametricity has been discussed and studied in the phylogenetics literature, it has not yet been acknowledged in the social network literature. We show that "non-ultrametric structure" in the network introduces significant instabilities in the existing top-down recovery algorithms. To address this issue, we introduce an instability diagnostic plot and use it to examine a collection of empirical networks. These networks appear to violate the "ultrametric" assumption. We propose a deceptively simple and yet general class of probabilistic models called $\mathbb{T}$-Stochastic Graphs which impose no topological restrictions on the latent hierarchy. To illustrate this model, we propose six alternative forms of hierarchical network models and then show that all six are equivalent to the $\mathbb{T}$-Stochastic Graph model. These alternative models motivate a novel approach to hierarchical clustering that combines spectral techniques with the well-known Neighbor-Joining algorithm from phylogenetic reconstruction. We prove this spectral approach is statistically consistent.