Statistical power and prediction accuracy in multisite resting-state fMRI connectivity

TL;DR

This study assesses the impact of multisite variability on resting-state fMRI connectivity analysis, showing that while inter-site effects are moderate, large sample sizes can mitigate their influence on statistical power and prediction accuracy.

Contribution

It provides empirical evidence and simulations demonstrating the feasibility of multisite rs-fMRI studies despite inter-site effects, highlighting the importance of sample size.

Findings

Inter-site effects are small to moderate with Cohen's d below 0.5.

Multisite variability slightly reduces detection power in group comparisons.

Prediction accuracy is more affected by multisite heterogeneity than statistical detection.

Abstract

Connectivity studies using resting-state functional magnetic resonance imaging are increasingly pooling data acquired at multiple sites. While this may allow investigators to speed up recruitment or increase sample size, multisite studies also potentially introduce systematic biases in connectivity measures across sites. In this work, we measure the inter-site effect in connectivity and its impact on our ability to detect individual and group differences. Our study was based on real, as opposed to simulated, multisite fMRI datasets collected in N=345 young, healthy subjects across 8 scanning sites with 3T scanners and heterogeneous scanning protocols, drawn from the 1000 functional connectome project. We first empirically show that typical functional networks were reliably found at the group level in all sites, and that the amplitude of the inter-site effects was small to moderate, with a Cohen's effect size below 0.5 on average across brain connections. We then implemented a series of Monte-Carlo simulations, based on real data, to evaluate the impact of the multisite effects on detection power in statistical tests comparing two groups (with and without the effect) using a general linear model, as well as on the prediction of group labels with a support-vector machine. As a reference, we also implemented the same simulations with fMRI data collected at a single site using an identical sample size. Simulations revealed that using data from heterogeneous sites only slightly decreased our ability to detect changes compared to a monosite study with the GLM, and had a greater impact on prediction accuracy. Taken together, our results support the feasibility of multisite studies in rs-fMRI provided the sample size is large enough.