Dismantling the Mantel tests

TL;DR

This paper critically examines the validity of Mantel and partial Mantel tests in evolutionary biology, revealing biases and limitations, especially with spatial autocorrelation, and suggests alternative methods.

Contribution

It demonstrates that partial Mantel tests are invalid for structured data and highlights the biases in their application, proposing alternative strategies.

Findings

Partial Mantel tests are biased and invalid for structured data.

Strong biases occur under typical sampling and spatial correlation conditions.

Mantel tests should not be used when autocorrelation is suspected.

Abstract

The simple and partial Mantel tests are routinely used in many areas of evolutionary biology to assess the significance of the association between two or more matrices of distances relative to the same pairs of individuals or demes. Partial Mantel tests rather than simple Mantel tests are widely used to assess the relationship between two variables displaying some form of structure. We show that contrarily to a widely shared belief, partial Mantel tests are not valid in this case, and their bias remains close to that of the simple Mantel test. We confirm that strong biases are expected under a sampling design and spatial correlation parameter drawn from an actual study. The Mantel tests should not be used in case auto-correlation is suspected in both variables compared under the null hypothesis. We outline alternative strategies. The R code used for our computer simulations is distributed as supporting material.