Editorial: Variability and reproducibility of brain imaging

Abstract

Full text

The human brain processes a constant stream of sensory information, necessitating advanced filtering and cognitive processing for tasks like recognizing dynamic facial expressions (Wang and Yuan, 2021). Advances in noninvasive neuroimaging methods, such as electroencephalogram (EEG) (Wang et al., 2020a), magnetic resonance imaging (MRI) (Lu et al., 2021), and near-infrared spectroscopy (NIRS) (Wang et al., 2020b) have deepened our understanding of how the human brain works over the last decades (Finn et al., 2023). However, the reliability and reproducibility of the results have been recently questioned and exposed in the spotlight (Cognitive neuroscience at the crossroads, 2022), highlighting the need to address and clarify the integrity of the research field (Revisiting doubt in neuroimaging research, 2022).

Hence, the main goal of this Research Topic is to address and explore which factors influence the reliability and reproducibility of brain imaging results and provide practical perspectives and insights to enhance them. We have collected four articles for the Research Topic, whose contributions were discussed as follows.

The first article explored the contributing factors to the brain age index. Brain age is the age gap between chronological age and predicated age from brain imaging data (Cole and Franke, 2017). There is evidence showing that brain age is associated with common brain disorders (Kaufmann et al., 2019), and it has been considered as a potential biomarker for various psychiatric disorders (Cole and Franke, 2017). However, the contribution to brain age from non-brain imaging factors besides neuroimaging data has not been explored. Korbmacher et al. have explored this issue and proposed a Bio-psycho-social model for predicting brain age. The study utilized the UK Biobank data (Miller et al., 2016) and compared different model configurations. They discovered that the Bio-psycho-social model can partially explain the brain age variance which is comparable to the contribution of the diffusion MRI approach. Additionally, they found large variability in gender and ethnicity differences in brain age. They then suggest that in future studies, the effects of ethnicity, cognitive factors, gender, as well as health and lifestyle factors should be controlled to observe the bio-psycho-social factor impacts on brain age.

The second article touched upon the reliability of the resting-state (rs-)fMRI. Put simply, rs-fMRI captures systematic, non-random variations in brain activation in the absence of a specific task. These activations are indirectly reflected in regional fluctuations in the level of oxygenation of the blood (known as BOLD) (Ogawa et al., 1992), which has been widely used to explore the functional brain activation (Lu et al., 2020) and brain networks (Damoiseaux et al., 2006; Bullmore and Sporns, 2009; van den Heuvel and Pol, 2010). There is evidence suggesting that the time-of-day could be associated with reduced global signal fluctuation and functional connectivity (Orban et al., 2020) alongside the time-of-year effect (Wang et al., 2023; Zhang et al., 2023). However, it's still unclear how the time-of-day effect, along with other factors such as age and gender, could sway the results of longitudinal neuroimaging data. Vaisvilaite et al. addressed this issue by leveraging the BETULA dataset, a longitudinal study on aging (Nilsson et al., 1997), and employing the cross-spectral dynamic causal modeling (DCM) (Friston et al., 2003), which puts the temporal fluctuations of resting-state BOLD signal into the focus. Specifically, they explored eight DCM models and found that the time-of-day effect together with gender significantly influences the parameters defining the BOLD signal. Therefore, they suggest that when collecting longitudinal neuroimaging datasets, time of day should be considered a covariate in addition to age and gender.

The third article discussed the reliability of the magnetic resonance spectroscopy (MRS) measurement in multi-site/vendor studies. MRS allows for non-invasive investigation of neurometabolic profiles in health and disease, yielding metabolite estimates from a localized volume (Oz et al., 2014; Wilson et al., 2019). Widely used in clinical settings (Oz et al., 2014), MRS has recently been applied to identify IDH-mutated gliomas (Branzoli and Marjanska, 2020) and explore the association between behavior and MRS-derived neurotransmitters (Li et al., 2022). It is suggested that the MRS-derived metabolite estimates are reliable within the participants (Wang et al., 2024). In large-scale clinical studies, a multi-site/vendor design is often favored to achieve the necessary statistical power. However, the impact of different sites and scanners on metabolite estimates is frequently overlooked (Považan et al., 2020). La et al. investigated how to account for these differences in a pediatric concussion dataset comprising 545 short-TE MRS measurements, collected across six MRI scanners with two MRI vendors at five scanning sites. They used four general linear models and three linear mixed-effects models to investigate site- and vendor effects on the quantitative estimates of five major neuro-metabolites. They found that different analysis strategies of controlling site, vendor, and scanner in MRS data generated different results, from which they advocate that the ComBat harmonization for clinical MRS data (Bell et al., 2022) should be utilized to remove the site and vendor effects.

The fourth article is even more clinically related, exploring the repeatability of 2D and 4D flow MRI measurement of intracranial blood flow and pulsatility. Arterial stiffening impacts blood vessel compliance and regulation and is considered a contributing factor to various neurological conditions (Poels et al., 2012). To noninvasively examine arterial stiffening specifically the blood flow velocity, 2D phase-contrast MRI is dominantly used (McCauley et al., 1995). However, it can only measure a few vessels per scan. On the contrary, the 4D flow MRI can evaluate multiple vessels simultaneously, making it an attractive alternative (Terada et al., 2022). Yet, the repeatability, reliability, and consistency of 4D flow compared to 2D across intracranial vessels remain uncertain. In the study, Morgan et al. aimed to answer this scientific inquiry. They used various statistical methods including intra-rater reliability, inter-method conformity, and test-retest repeatability to assess the repeatability of the pulsatility index and mean flow among patients with small vessel disease and healthy controls. They found that the 4D flow MRI possesses repeatable and reliable PI measurements and the mean blood flow measurement. Therefore, they have suggested the 4D flow MRI can be confidently used to assess the pulsatility across the major cerebral vessels.

We appreciate the contributions of the authors of these four articles, which hopefully could stimulate and elicit further discussions about the topic: the variability and reliability of brain imaging. Indeed, we have witnessed collective initiatives to enhance the robustness of the neuroimaging results (Korbmacher et al., 2023), such as best practices in the data analysis (Nichols et al., 2017; Boudewyn et al., 2023), guidelines for reporting or publishing (Poldrack et al., 2008; Keil et al., 2014; Lin et al., 2021; Yücel et al., 2021), and expert consensus (Choi and Kreis, 2021). We firmly believe by acknowledging the problems we have in neuroimaging studies and adopting the best practices, the neuroimaging results will be more robust and reliable.

Author contributions

M-YW: Writing—original draft, Writing—review & editing, Conceptualization. HZ: Writing—review & editing, Conceptualization. MY: Writing—review & editing, Conceptualization. KS: Writing—review & editing, Conceptualization.