Misspecification in mixed-model based association analysis

TL;DR

This paper demonstrates that in mixed-model association analyses, population structure and epistatic interactions can cause inflated estimates of additive genetic variance, challenging common assumptions especially in related populations.

Contribution

It reveals how non-additive effects and relatedness can bias genetic variance estimates in mixed models, highlighting a key limitation in current methods.

Findings

Inflated additive genetic variance estimates occur with related individuals.

Population structure combined with epistasis affects variance estimates.

Assumption of residual-only effects from non-additive genetics is invalid in related samples.

Abstract

Additive genetic variance in natural populations is commonly estimated using mixed models, in which the covariance of the genetic effects is modeled by a genetic similarity matrix derived from a dense set of markers. An important but usually implicit assumption is that the presence of any non-additive genetic effect only increases the residual variance, and does not affect estimates of additive genetic variance. Here we show that this is only true for panels of unrelated individuals. In case there is genetic relatedness, the combination of population structure and epistatic interactions can lead to inflated estimates of additive genetic variance.