Analysis of comparative data with hierarchical autocorrelation

TL;DR

This paper investigates the asymptotic properties of estimators and model selection criteria in hierarchical autocorrelation models, especially in biological data, highlighting limitations of traditional methods and proposing corrections for effective sample size.

Contribution

It introduces a framework for understanding estimator convergence in hierarchical autocorrelation models and proposes adjustments to BIC for better model selection in such contexts.

Findings

Estimators are almost surely convergent but may not be consistent for some parameters.

Standard BIC is inadequate for model selection with hierarchical autocorrelation.

Effective sample size adjustments improve model selection accuracy.

Abstract

The asymptotic behavior of estimates and information criteria in linear models are studied in the context of hierarchically correlated sampling units. The work is motivated by biological data collected on species where autocorrelation is based on the species' genealogical tree. Hierarchical autocorrelation is also found in many other kinds of data, such as from microarray experiments or human languages. Similar correlation also arises in ANOVA models with nested effects. I show that the best linear unbiased estimators are almost surely convergent but may not be consistent for some parameters such as the intercept and lineage effects, in the context of Brownian motion evolution on the genealogical tree. For the purpose of model selection I show that the usual BIC does not provide an appropriate approximation to the posterior probability of a model. To correct for this, an effective sample size is introduced for parameters that are inconsistently estimated. For biological studies, this work implies that tree-aware sampling design is desirable; adding more sampling units may not help ancestral reconstruction and only strong lineage effects may be detected with high power.