Diffusion‐MRI‐Based Estimation of Cortical Architecture via Machine Learning (DECAM) in Primate Brains
Tianjia Zhu, Minhui Ouyang, Shufang Tan, Jianlin Guo, Ziqin Zhang, Xuan Liu, Risheng Liu, Hao Huang
TL;DR
DECAM is a machine learning framework that uses diffusion MRI to noninvasively map brain cortical architecture in primates, enabling virtual histology.
Contribution
DECAM introduces a novel deep learning framework optimized with best response constraint and cortical label vectors for accurate primate brain mapping.
Findings
DECAM generates high-fidelity, reproducible whole-brain soma density maps validated with histology.
The framework addresses dMRI-histology misregistration in complex primate brain morphology.
DECAM is generalizable for estimating other neuropathological measures in human brains.
Abstract
The cerebral cortical cytoarchitecture underlying brain functions is reshaped across the lifespan and in various brain disorders. Accumulated evidence indicates it is important to disease biology. The cortical cytoarchitecture is conventionally accessible only through invasive neuropathological techniques. Diffusion MRI (dMRI) holds the potential to reveal whole‐brain cytoarchitecture noninvasively. However, current dMRI signal models are constrained by simplified assumptions, which limit their ability to accurately quantify cortical architecture. Here, we present Diffusion‐MRI‐based Estimation of Cortical Architecture using Machine‐learning (DECAM), a cutting‐edge data‐driven translational framework capable of accurately and directly mapping the heterogeneous, whole‐brain soma density in primates. Leveraging high‐resolution multi‐shell dMRI and histological datasets of the non‐human…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
FIGURE 1
FIGURE 2
FIGURE 3
FIGURE 4
FIGURE 5
FIGURE 6
FIGURE 7
FIGURE 8- —National Institutes of Health10.13039/100000002
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neuroimaging Techniques and Applications · Functional Brain Connectivity Studies · Fetal and Pediatric Neurological Disorders
Introduction
1
The cerebral cortical cytoarchitecture provides the substrate for brain functional changes across the lifespan and in various brain disorders. The cerebral cortex is precisely regulated by elaborate genetic and molecular mechanisms [1, 2], yielding heterogeneous cortical architecture across regions. Cerebral cortical cellular architecture is immensely complicated. It roughly includes somas (e.g., cell bodies of neurons and glia), neurites (e.g., axons and dendrites projecting from the cell body), and extracellular matrix. Changes in cortical cytoarchitecture measures, such as densities of soma [3], synapse [1, 4], and dendritic spine [5, 6] were observed across the lifespan. Alterations in the spatiotemporal progression of cortical architecture have also been observed in various brain disorders, including autism [3, 5, 6], schizophrenia [5, 6], and Alzheimer's disease [7, 8, 9, 10]. Therefore, cortical architecture serves as an important brain lifespan biomarker, as well as a biomarker for the prognosis of various brain disorders. However, quantifying cortical architecture is challenging because of its complexity and heterogeneity. At present, the most widely used method for assessing cortical architecture is invasive neuropathology, using various stains such as Nissl for cell bodies, Golgi for whole neurons, and Weigert stain for myelinated fibers, to highlight different cells, somas, or neurites in the cortex. Invasive neuropathology provides only local neuronal architecture information and is not suitable for prognosis and longitudinal monitoring of disease progression. Developing a non‐invasive technique to accurately measure the complex whole‐brain cortical cytoarchitecture will change the current paradigm for precision and individualized medicine.
Diffusion MRI (dMRI) has been the method of choice for noninvasively measuring brain microstructure. Significant advancements have been made in quantifying the relationships between dMRI signal and the underlying brain cellular architecture using “top‐down” approach. In a typical “top‐down” approach, dMRI signals are modeled or represented through mathematical formulas that simplify the complex real‐world brain cellular architecture into regularized geometrical shapes such as ellipsoids, balls, and sticks. For example, diffusion tensor imaging (DTI) [11] is the most conventional dMRI signal representation, primarily used for quantifying white matter microstructure [12]. DTI has also been used to characterize cortical microstructure [13, 14], especially for early developing brains [15, 16, 17, 18, 19, 20, 21, 22, 23, 24]. Other state‐of‐the‐art dMRI signal representations, such as diffusion kurtosis imaging (DKI) [25], characterizing fourth‐order tensor and more sophisticated biophysical models [26, 27, 28, 29], have been used to assess cortical microstructure [30, 31, 32, 33]. However, such “top‐down” signal representations and models incorporate relatively strong assumptions and simplifications of underlying cellular architecture, and possibly result in significant deviation of measured microstructure from the ground‐truth. In contrast to the “top‐down” approach, the deep learning (DL)‐based, “bottom‐up” data‐driven methods have an inherently unparalleled ability to delineate highly complex mappings between dMRI signals and cortical cytoarchitecture, offering a promising solution to the challenge of estimating cortical architecture from dMRI. DL‐based frameworks do not require any assumptions to simplify the ground‐truth cellular architecture. The cytoarchitectural measures obtained from DL‐based frameworks can also be validated against those from neuropathology, enabling accurate noninvasive virtual histology. Most existing applications of DL to diffusion MRI mainly focus on enhancing existing signal representations or model estimations without directly learning histology‐derived cortical architecture from dMRI [34]. Recent progress has been made in estimating certain neuropathological features [35, 36] using MRI through machine learning. However, these estimations are either indirect neuropathological measurements for anomaly detection or histological staining intensities that could vary with staining methods. These prior studies were only conducted on mouse brains or pieces of human brain gyri with relatively less complicated morphology. As the non‐human primate brain is characterized by a more heterogeneous cortical architecture closer to humans and a more complex morphology in comparison to the rodent brain, a DL‐based framework for directly estimating whole‐brain cortical cytoarchitecture measures using dMRI of primate brains is hence needed to pave the way for noninvasive virtual histology in translational applications.
In this study, we developed a novel technique called Diffusion‐MRI‐based Estimation of Cortical Architecture via Machine learning (DECAM) (Figure 1) to provide a direct mapping from dMRI signals to critical soma density (SD) measures for quantifying cortical architecture in the non‐human primate brains. Unlike the traditional invasive approach demonstrated in the light blue semicircle in Figure 1, DECAM tackles the challenge of non‐invasively estimating high‐fidelity whole‐brain cortical cytoarchitecture in primate brains, illustrated in the dark blue circle in Figure 1. DECAM trains and optimizes a state‐of‐the‐art generative adversarial network (GAN) [37] by incorporating the innovative best response constraint (BRC) [38] learning framework and cortical label vectors. The multi‐shell dMRI signals capture the non‐Gaussian and restrictive diffusion arising from cortical cellular architecture, including somas and neurites. The established DECAM framework successfully generated high‐fidelity cortical SD maps in macaque brains, as validated by ground‐truth SD quantified from Nissl‐stained histology. DECAM also revealed the heterogeneous cellular architecture across the macaque cortex, and such an inhomogeneous spatial pattern was reproducible across different macaques. Collectively, we developed a DECAM framework by incorporating innovative BRC‐GAN and cortical label vectors and highlighted high‐fidelity and reproducible estimation of whole‐brain cortical architecture in primates with DECAM.
Schematic representation of the proposed DECAM framework for primate brain. Dark blue circle demonstrates DECAM‐enabled virtual neuropathology. DMRI images concatenated with cortical labels vectors and paired with registered ground‐truth soma density maps from Nissl‐stained histology images were fed into a deep learning framework optimized with best response constraint (BRC) to train the framework. The trained framework can be used for estimating non‐invasive neuropathology on new brain datasets. The light blue semicircle demonstrates quantifying neuropathology through an invasive approach. After brain extraction, the macaque brain was sectioned, stained, and scanned, yielding high‐resolution histological images. Soma density was calculated from the processed and digitized high‐resolution Nissl‐stained histology images and registered to dMRI to serve as ground‐truth data for training and validating DECAM. Created in BioRender Zhu, T. (2025) https://BioRender.com/01ie2ms.
Results
2
DECAM Leverages the Sensitivity of Multi‐Shell Diffusion MRI Signals to Cortical Cellular Architecture for Its Precise Estimation
2.1
Cortical microstructure at the micrometer (µm) scale is heterogeneous across the cerebral cortex, as shown in Figure 2a. In a typical dMRI pulse sequence, a regular range of diffusion time of 10–100 ms [39] corresponds to a diffusion distance of water molecules at the level of µm or tens of µm. These distance scales are at the same level of scale of individual soma of roughly 5–100 µm in diameter. In regions with a relatively higher density of cytoarchitecture, water molecules diffuse over shorter distances. This shorter distance diffusion results in smaller phase differences in the dMRI signal (Figure 2b top panel in the orange box), thereby resulting in a smaller signal drop. In contrast, in regions with a relatively lower density of cytoarchitecture, water molecules diffuse over longer distances. This longer diffusion distance leads to larger phase differences in the dMRI signal (Figure 2b middle panel in the green box) and, consequently, a more pronounced signal drop. When dMRI is acquired with multiple diffusion weightings, e.g., multi‐shell dMRI, the signal drop patterns across cortical regions become sensitive indicators of the cellular architecture at the level of µm (Figure 2b bottom panel). DECAM harnesses this sensitivity by directly learning the mapping between multi‐shell dMRI signals and the underlying cortical cellular architecture. Unlike “top‐down” approaches, which adopt restrictive signal models that may not capture the true complexity of cortical microstructure (Figure 2c, left panel), DECAM utilizes a “bottom‐up” strategy directly translating the intricate dMRI signal variations into high‐fidelity estimates of cortical architecture. Figure 3 outlines the detailed DECAM DL framework. Soma density maps quantified from high‐quality Nissl‐stained histology data (Figures S1 and S2) were used as ground‐truth in training and validating DECAM. Registered high‐resolution multi‐shell dMRI (Figure S3) and SD maps derived from histology datasets (Figure 3a) serve as training data for the DL framework (Figure 3b) to estimate SD. Cortical label vectors mitigate misregistration from the complex morphology of the primate brain. The presented DL algorithm employs a GAN framework optimized by a Best‐Response‐Constraint module (BRC‐GAN), which explicitly couples the GAN generator and discriminator networks. DECAM can also be extended for non‐invasively estimating cortical architecture maps given multi‐shell dMRI datasets of new macaque brains.
Sensitivity of multi‐shell diffusion MRI signals to cortical cellular architecture underlies the DECAM framework. a Heterogeneous microstructure across entire cerebral cortex in a macaque brain. Close‐up views at 1 mm and 100 µm scales highlight regional heterogeneity. Orange box denotes cortical region with higher soma density, while green box indicates region with lower soma density. b Voxel‐averaged multi‐shell diffusion MRI (dMRI) signals at mm scale are sensitive to cortical microstructure barriers at µm scale. Top and middle panels: Shorter and longer water molecule diffusion distances correspond to smaller and larger phase difference in dMRI signals, demonstrated in orange and green boxes, respectively. Bottom panel: DMRI signals at the mm scale represented by diffusion‐weighted images are shown on the left; Signal drop curves across b values quantifying diffusion‐weighting, with orange/green lines representing signal drop corresponding to shorter/longer diffusion distances, are shown on the right. The gray line in the bottom right plot denotes linear signal drop in cerebrospinal fluid (CSF), reflecting free diffusion. c Top‐down and bottom‐up approaches for measuring cortical microstructure. The top‐down approach incorporates simplified signal model equations with restrictive assumptions and may not authentically represent the ground‐truth microstructure. Bottom‐up approach directly learns a mapping between dMRI signal and ground‐truth microstructure via data‐driven deep learning framework such as DECAM.
DECAM Deep learning framework optimized with best response constraint (BRC). a Top and middle panels: Registration of histology to diffusion MRI (dMRI), and incorporation of cortical label vectors into the deep learning framework to correct dMRI‐histology misregistration. Bottom panels: Procedures of quantifying soma density (SD) with Nissl‐stained histology images. High‐resolution (0.46 µm/pixel) Nissl‐stained histology image was segmented, converted to gray scale, and threshold. SD is defined as the number of contoured units in the image divided by the segment area. b Deep learning framework of the best response constraint generative adversarial network (BRC‐GAN) with cortical labels vectors. In the GAN architecture, the generator consists of residual blocks and convolution layer (Conv), while the discriminator consists of Conv, leaky rectified linear unit (Leaky ReLU), batch normalization (BN), and sigmoid layers. The BRC is a constraint S(θ G), that formulates the dependency of the generator parameters θ G on the discriminator parameters θD to optimize the generator cost function F(θ G, θ D(θ G)) in the training procedure. f(θ G, θ D) is the discriminator cost function. c Representative slices of the estimated SD map.
Cortical Label Vectors Resolve Challenges in Diffusion MRI‐Histology Misregistration due to Complex Primate Brain Cortical Morphology
2.2
Accurate estimation of cortical architecture relies heavily on precise registration between histology and dMRI images. However, given the complex cortical morphology and individual variability of primate brains, misregistration between histology and dMRI often occurs. Histology images were mapped to dMRI space by transforming gray‐scaled Nissl‐stained images into averaged diffusion‐weighted image using 12‐parameter affine transformation followed by a non‐linear large deformation diffeomorphic metric mapping (LDDMM) transformation [40], resulting in co‐registered histology and dMRI in the dMRI space. Subsequently, macaque brain digital atlas [41] labels were mapped to dMRI space with the transformation driven by dMRI contrast of the digital atlas and subject dMRI. A prominent difference of the gyral label, indicated by a green contour, is demonstrated in Figure 4a with the left superior parietal lobule (SPL‐L) label as an example. The same set of landmarks indicated by red dots placed based on histology SPL‐L contour is overlaid on dMRI. The mismatching contour was indicated by the landmarks pointed by gray arrows. Training a DL framework on misregistered data can erroneously map a dMRI voxel in one cortical region to a histology voxel in another cortical region, probably with distinctive cortical cytoarchitecture, leading to inaccurate estimations of cortical cytoarchitecture. To resolve the misregistration, we created the cortical label vector by extracting the overlap area of a specific cortical label transferred from the macaque brain atlas in co‐registered dMRI and histology images, representing the location of well‐registered voxels in a particular structure (Figure 4b). For example, a specific cortical region would have ones in its corresponding label location, and zeros elsewhere (Figure 4c). We then one‐hot encoded cortical label vectors in K channels with K equal to the number of cortical regions, and fed them into the DL framework as inputs along with the dMRI data. Hence, each channel informs the DL framework of the locations of well‐registered voxels from a specific cortical region. The purpose of introducing one‐hot encoding represented by cortical label vectors is to inform the deep learning framework which voxels are well‐registered. Using the cortical label vectors ensures the DL framework learns only from well‐registered voxels. To demonstrate the effectiveness of cortical label vectors in improving estimation results, we compared GAN frameworks with and without cortical label vectors. GAN without cortical label vectors was trained using all cortical voxels, instead of well‐registered, overlapping voxels. When tested against GAN without cortical label vectors, GAN with cortical label vectors significantly improved estimation accuracy, yielding higher correlations (t = 3.25, *p_FDR_
- = 0.034 with FDR standing for false discovery rate) between estimated and ground‐truth SD across slices (Figure 5). As demonstrated in Figure S4, GAN was selected as the optimal baseline DL framework since GAN outperformed a few other widely used patch‐based DL frameworks, including ResNet and UNet, in SD estimation, and achieved the lowest residual and highest r values.
Cortical label vectors resolving challenges of dMRI‐histology misregistration in macaque brains due to complex morphology. a After mapping the cortical labels using digital macaque atlas [41] shown on the left, cortical regions such as left superior parietal lobule (SPL‐L) were delineated on the diffusion MRI (dMRI) and Nissl‐stained histology. Red dots indicating anatomical landmarks of SPL‐L were delineated based on the contour of SPL‐L on histological image. The same set of landmarks were overlaid on dMRI. Despite linear registration and nonlinear warping, due to the complex morphology of macaque brains, misregistration of SPL‐L between dMRI and histology remains prominent as indicated by the red dots with identical coordinates pointed by gray arrows. b Extraction of dMRI data of the same cortical label from overlapped cortical regions in dMRI and histology in a. c One hot encoding of cortical label vectors associated with extracted dMRI data of the same cortical label in b as input in DECAM deep learning framework.
*Precise estimation of cortical cytoarchitecture using GAN optimized by BRC and cortical label vectors. a Residual maps between estimated and ground‐truth soma density (SD) for six coronal slices. The estimated SD maps were obtained with four different frameworks BRC‐GAN with cortical labels vectors adopted by DECAM (green), BRC‐GAN without cortical label vectors (purple), GAN with cortical label vectors (blue), and GAN (red). b Scatterplot showing the Pearson correlation coefficient (r) and mean residual for SD estimation from the same four frameworks specified in panel a, with high r values associated with low residuals. Each dot represents individual slices. Boxplots show the distribution of r values (right axis) and residuals (top axis) across four frameworks between ground‐truth SD and estimated SD. The centers of boxes are median values. Bounds of boxes are first and third quartile, with ends of whiskers representing the minima and maxima of non‐outlier points of the distribution. Statistical significance of mean r value and mean residual across models was assessed using repeated‐measures one‐way analysis of variance (ANOVA) followed by paired t‐tests after correction for false discovery rate (FDR). Asterisks indicate significance: *0.01 ≤ pFDR < 0.05, *pFDR < 0.01. Same colors encoding four frameworks as those in panel a were applied. c Histograms with boxplots of voxel‐wise differences between estimated and ground‐truth SD across the four deep learning (DL) frameworks with colors of the DL framework matching those in panel a and b. Black dots indicate the mean value of voxel‐wise differences between ground‐truth and estimated SD from each DL framework. Black dashed line indicates location of zero difference.
Optimized GAN With Best Response Constraint (BRC) for Precise Estimation of Cortical Cytoarchitecture
2.3
While cortical label vectors improve registration accuracy, further optimization of the presented deep learning framework requires addressing the limitations of traditional GAN frameworks. We introduce the BRC‐GAN, a novel bi‐level optimization technique that effectively couples generator and discriminator parameters to enhance training stability and accuracy. In traditional single‐level minimax GANs, the generator and discriminator loss functions are updated alternatively and independently. Such an optimization procedure cannot exactly reveal the coupled relationship between the generator and discriminator, leading to vanishing gradients and unstable training. BRC‐GAN resolves this by modeling the dependency between the generator and discriminator through bi‐level optimization, adding a discriminator gradient‐based term to the generator's gradient during optimization, as illustrated in the blue box of Figure 3b and pseudocode in Methods. Figure 5a demonstrates that incorporating BRC into the GAN framework with cortical label vectors further reduced residuals between estimated and ground‐truth SD across six representative slices. One‐way ANOVA confirmed a significant effect of model type on both correlation (F(3,20) = 15.48, p = 0.001) and residuals (F(3,20) = 14.71, p = 0.001). Compared to the traditional GAN, BRC‐GAN with cortical label vectors achieved significantly higher correlations (r = 0.69 ± 0.08; t = 4.91, *p_FDR_
- = 0.027) and lower MAE (0.020 ± 0.001; t = ‐9.94, *p_FDR_
- = 0.001). Even without cortical label vectors, BRC‐GAN significantly outperformed the traditional GAN in both correlation (t = 3.92, *p_FDR_
- = 0.034) and MAE (t = 6.58, *p_FDR_
- = 0.004), though its performance remained below that of BRC‐GAN with cortical label vectors. A direct comparison of BRC‐GAN with and without cortical label vectors further demonstrated the added value of cortical label vectors, with the MAE trending lower (t = −2.69, *p_FDR_
- = 0.065) when cortical label vectors were included. These findings together indicate that both cortical label vectors and the BRC independently enhance estimation accuracy, with the combination of the two yielding the most precise estimations of cortical cytoarchitecture. These improvements with cortical label vectors and BRC are visually represented by the green scatter points in the upper left corner of Figure 5b, as well as the corresponding box plots in the top and right panels. Figure S5 shows the scatter plots and linear correlations between the estimated SD from all the above‐mentioned four DL frameworks and the ground‐truth SD across slices. We further examined the residual profile of all voxels within these slices. Figure 5c shows histograms that highlight the differences between the estimated and ground‐truth SD. Notably, BRC‐GAN with cortical label vectors exhibited a residual distribution with values closer to zero in the cortex (mean difference = 0.002 ±0.032) compared to the other two GAN frameworks (BRC‐GAN: mean difference = −0.007 ± 0.030; GAN with cortical label vectors: mean difference = −0.009 ± 0.034; GAN without cortical label vectors: mean difference = −0.022 ± 0.036).
High Fidelity of DECAM in Estimating Cortical Cytoarchitecture
2.4
We demonstrated the consistent heterogeneous cytoarchitectural pattern between the DECAM‐estimated SD map and the histology image in Figure 6a. Specifically, enlarged light blue boxes highlighting the SPL‐R cortex show consistently higher SD in both the DECAM‐estimated SD map and the histology image. In contrast, enlarged black boxes highlighting the parahippocampal cortex demonstrate consistently lower SD in both images of the same slice. We further demonstrated the high fidelity and reliability of DECAM by using leave‐one‐out cross‐validation of cortical SD estimations against ground‐truth histology across multiple levels, from individual slices (Figure 6b) to cortical gyral levels (Figure 6c). The scatter plots in Figure 6b reveal significant, high correlations (r values range from 0.56 to 0.8, all p ≤ 1×10^−25^) between estimated and ground‐truth SD, with the scatterplot for the slice indicated by the blue vertical line enlarged (r = 0.80, p = 1 × 10^−30^). At the cortical gyral level, the gyral‐averaged SD values estimated by DECAM show a strong and significant correlation (r = 0.73, p = 7.76 × 10^−8^) with the ground‐truth measurements (Figure 6c). Here, scatter points are color‐coded by cortical lobes, reflecting the heterogeneity of SD across different brain lobes and gyri. Table S1 lists all the cortical gyri presented. Permutation tests confirmed the significance of the correlations at both individual slices and cortical gyrus (all p perm < 0.001), highlighting the high fidelity of the DECAM‐estimated cortical architecture at slices and gyri.
High fidelity and reliability of DECAM in estimating cortical cytoarchitecture across slices and gyral levels. a Comparison of estimated SD (left panel) with Nissl‐stained histology images (right panel) in a representative coronal slice. Its anatomical location is indicated by vertical blue line on the midsagittal image of a macaque brain serving as anatomical reference on the lower left corner. Enlarged light blue boxes show the right superior parietal lobule with higher SD consistent with denser Nissl staining. Enlarged black boxes show right parahippocampal gyrus with lower SD consistent with sparser Nissl staining. b The density plots show distribution of DECAM‐estimated SD (y‐axis) against the histologically derived ground‐truth SD (x‐axis) for various coronal slices. Each data point is represented by a hexagonal binning with colors encoding the frequency of data points within each hexagon. The anatomical locations of the coronal slices are indicated by vertical lines on the midsagittal image of a macaque brain serving as anatomical reference on the lower left corner. The enlarged main bigger distribution map corresponding to the vertical blue line demonstrates that DECAM‐estimated SD significantly correlated with the ground‐truth SD (r = 0.80, p = 1 × 10−30; Pearson correlation; two‐sided). A white dashed line shows the linear fit of estimated versus ground‐truth values. c Significant correlation between estimated and ground‐truth average gyral SD (r = 0.73, p = 7.76 × 10−8; Pearson correlation; two‐sided), with each point representing estimated and ground‐truth gryal SD from each parcellated gyrus defined by the digital macaque atlas [41] across different lobes. The underlying gray points of SD measures of frontal, temporal, parietal and occipital lobe gyri and the correlation lines are the same across the four panels in c. In each panel, frontal, temporal, parietal and occipital lobe gyri are highlighted by blue, orange, green, and red, respectively. Abbreviation of each cortical gyrus is labeled in each subpanel of c. L: left, R: right. Full names of cortical regions are listed in Table S1.
When the estimated SD is projected to the cortical surface, BRC‐GAN with cortical labels vectors produced a heterogeneous SD map (Figure S6, top right panel) that closely matches the ground‐truth cortical SD quantified from histology (Figure S6, left panel) and is consistent with previous studies [42, 43, 44]. Specifically, black arrows highlight consistently higher SD along the precentral and superior temporal gyrus in both estimated and ground‐truth SD. In contrast, a more homogeneous SD map across the cortex was generated with less optimized frameworks, such as GAN, without cortical label vectors (Figure S6, bottom right panel), deviating from the ground‐truth SD. The DECAM framework incorporating BRC‐GAN and cortical label vectors demonstrates high fidelity in estimating cortical cytoarchitecture on the reconstructed surface.
High Reproducibility of Cortical Cytoarchitecture Profiles Across Multiple Macaque Brains
2.5
We further tested the reproducibility of DECAM estimation of cortical cytoarchitecture for two additional macaque brains (macaques #2‐3) that were not used during DECAM training after testing fidelity and reliability on the training brain (macaque #1). As shown in Figure 7, the estimated whole‐brain SD maps reveal a reproducible inherent SD heterogeneity throughout the cerebral cortex across the macaques. Specifically, Figure 7a demonstrates high heterogeneity of the estimated SD maps that aligns with known neuroanatomy, with consistently higher SD values observed in the precentral gyrus (blue arrows), superior frontal gyrus (pink arrows), and superior and middle temporal gyrus (black arrows) across all three macaques. The spatial profiles of cortical SD along the anterior‐posterior axis of the two representative cortical slabs outlined by pink and green dashed lines in Figure 7a were also quantitatively revealed to further assess across‐subject reproducibility, as shown in the ridgeline plots in Figure 7b. The SD distributions normalized to the range of 0–1 and along these slabs for each macaque are illustrated in Figure 7b, with red or green color gradients indicating SD values. Across all three macaques, the SD profiles exhibit similar peak and valley locations along the anterior‐posterior axis, reflecting a consistent pattern of SD variation (Figure 7b). In Figure 7c, pairwise correlations of gyral SD among three macaques are high and significant (all r values range from 0.77 to 0.85, all p values ≤ 1.33×10^−12^; Pearson correlation), further confirming the reproducibility of SD distribution patterns across subjects. Permutation tests confirmed that all pairwise correlations were highly significant (all p perm < 0.001). These results suggest that DECAM‐estimated SD maps are highly reproducible across macaque brains, highlighting DECAM's reliability in revealing cortical SD heterogeneity.
Reproducibility of estimated soma density (SD) maps across three macaque brains. a Estimated whole‐brain SD maps across three macaque brains. Blue, pink, and black arrows indicate consistently higher SD at the precentral gyrus, superior frontal gyrus, and superior temporal gyrus, respectively across three subjects. A: anterior, P: posterior, L: left, R: right. b Ridgeline plots demonstrating profile of estimated left and right brain cortical SD along the representative slabs shown in a and indicated respectively by red and green color gradients for each subject. c Significant correlations between estimated and ground‐truth average gyral SD of subject pairs (n = 66, all r ≥ 0.77, p ≤ 1.33 × 10−12, Pearson correlation; two‐sided), with parcellated gyri defined by the digital macaque atlas [41]. Abbreviation of each cortical gyrus is labeled in the top panel of c. L: left, R: right. Full names of parcellated cortical gyri are listed in Table S1.
DECAM Outperforms dMRI Signal Models in Estimating Cortical Microstructure
2.6
To assess the performance of DECAM relative to dMRI signal models in estimating cortical microstructure, we evaluated the correlation between estimated SD or metric measures and ground‐truth SD using density plots (Figure 8). Each density plot visually represents the distribution of DECAM‐1 trained on macaque #1 or dMRI signal model metric measures against the ground‐truth SD values, with a fitted linear regression line indicating the strength and direction of these correlations. Pearson correlation coefficients (r) were calculated to evaluate the performance of DECAM and various dMRI signal models. Although DECAM‐1 and all dMRI signal models showed significant correlations confirmed by permutation tests (all p perm < 0.001), DECAM‐1 achieved the highest correlation (r = 0.55, p = 1 × 10^−30^; Figure 8a), signifying the strongest agreement with ground‐truth SD among all evaluated metrics (Figure 8b). In contrast, metrics derived by dMRI signal models, such as DTI‐derived fractional anisotropy (FA) and mean diffusivity (MD), DKI‐derived mean kurtosis (MK), and neurite orientation dispersion and density imaging (NODDI)‐derived [29] intra‐cellular volume fraction (ICVF), demonstrated lower r values, indicating weaker correlation with the ground‐truth SD. Estimated metric measures from DECAM and dMRI‐signal models were also mapped onto the cortical surface (Figure S7a). Using the cortical gyral parcellation (Figure S7b) of a representative slice from the digital macaque atlas [41], these measures were normalized into the range of 0–1 and demonstrated with violin plots in Figure S7c. Normalized DECAM‐estimated SD distributions closely match the normalized ground‐truth SD distributions, whereas the normalized dMRI‐signal model metric measures deviate from the normalized ground‐truth SD distributions across cortical gyri (Figure S7c). These findings suggest the irreplaceable capability of DECAM in accurately capturing SD profiles across cortical regions.
DECAM outperforms metric measures derived from diffusion tensor imaging (DTI), diffusion kurtosis imaging (DKI), and neurite orientation dispersion and density imaging (NODDI) in accuracy for estimating soma density (SD). a The density plot shows distribution of DECAM‐estimated SD (y‐axis) of all slices against the histologically derived ground‐truth SD (x‐axis). Each data point is represented by a hexagonal binning with colors encoding the frequency of data points within each hexagon. DECAM‐estimated SD of all slices significantly correlated with the ground‐truth SD (n = 47128, r = 0.55, p = 1 × 10−30; Pearson correlation; two‐sided). A white dashed line shows the linear fit of estimated versus ground‐truth values. b Absolute Pearson correlation coefficient between DECAM‐estimated SD and ground‐truth SD as well as absolute Pearson correlation coefficient between diffusion‐MRI‐signal‐model‐estimated metric measures and ground truth SD. The density plot was displayed for each correlation. In each density plot, a white dashed line shows the linear fit of estimated versus ground‐truth value. Abbreviations: FA: fractional anisotropy, MD: mean diffusivity, MK: mean kurtosis, ICVF: intra‐cellar volume fraction.
To further assess DECAM robustness and reproducibility across subjects, besides training DECAM‐1, we trained two other independent DECAM model versions (DECAM‐2 and DECAM‐3) on macaque brains #2 and #3, respectively, using the same approach of training DECAM‐1. The three model versions were validated on all three macaque brains by evaluating the correlation between estimated SD and ground‐truth SD. Based on the results of the three model versions, we also performed a more rigorous comparison of DECAM model versions trained on different subjects against dMRI signal models. Figure S8 demonstrates that each DECAM model version performed consistently across subjects and, importantly, all three DECAM model versions robustly outperformed dMRI‐derived metrics, including FA and MD from DTI, MK from DKI, and ICVF from NODDI. The boxplots show that DECAM does not depend on which macaque brain was used for training, providing direct evidence for both the reproducibility and generalizability of the framework.
Discussion
3
In this study, we developed DECAM, a cutting‐edge DL‐based framework that addresses the challenge of noninvasive estimation of whole‐brain cortical cytoarchitecture with high‐fidelity and reproducibility in primate brains using dMRI. DECAM can effectively and reliably capture the heterogeneous cellular architecture in entire primate brains with complex morphology. We demonstrated high‐fidelity of DECAM‐estimated SD maps validated by ground‐truth histological SD and showed highly reproducible whole‐brain heterogeneous cortical SD patterns across multiple brains. The proposed framework is generalizable. DECAM can be further extended for noninvasively estimating other neuropathology measures, such as neurite density, and extended for estimating neuropathology measures in human brains. Collectively, DECAM offers high‐fidelity, reproducible whole‐brain neuropathology maps validated by histology, getting closer to noninvasive virtual histology for studying brain disorders across the lifespan and advancing precision medicine.
Cortical cytoarchitecture serves as a critical biomarker across the lifespan [5, 10] and in various disorders [3, 5, 6, 7, 8, 10]. Evidence from prior studies (see review [5]) suggests that deviations from normal cortical cytoarchitecture can occur well before cognitive or behavioral symptoms become evident. These prior findings underpin the potential of whole‐brain longitudinal spatiotemporal mapping of cytoarchitecture as a powerful tool for digital prognosis, diagnosis, and monitoring of various neurological conditions. Traditional histology, despite its high resolution, is invasive and local, and cannot track cortical cytoarchitecture longitudinally. Noninvasive dMRI has the potential to lift this limitation. By leveraging the sensitivity of dMRI signals to the underlying cortical cytoarchitecture, dMRI can noninvasively assess the cortical cytoarchitecture. In the “top‐down” approach used for dMRI signal modeling [11, 25, 26, 27, 28, 29], mathematical formulas of the dMRI signals are derived from a simplified physical representation of the diffusion process in the underlying microstructure. To make the Bloch‐Torrey equation governing dMRI signals solvable, these models incorporate simplifying assumptions [39, 45, 46] to create manageable boundary conditions and solve the feasible parameter estimation problems. For example, these models often approximate somas as perfect, impermeable spheres and neurites as idealized, stick‐like structures. Although these assumptions simplify mathematics, they fail to capture the complexity of real‐world cortical cellular architecture, consisting of irregular compartments with membranes of variable permeability, making it challenging to precisely measure cortical microstructure information such as SD. As a non‐invasive “bottom‐up” computational framework leveraging DL and using dMRI to substitute invasive histology for accurately quantifying cortical architecture, DECAM provides an unprecedented opportunity for longitudinal monitoring of cortical cytoarchitecture for precision and individualized medicine. By directly mapping dMRI signals to the complex ground‐truth cortical cytoarchitecture using DL, DECAM bypasses the need for oversimplified physical assumptions about the microstructure and captures the intricate relationships between dMRI signals and cortical architecture more accurately and effectively than “top‐down” methods. From this perspective, the data‐driven, “bottom‐up” approach offers a significant advantage over “top‐down” approach. Although “top‐down” signal models could be further developed, such “bottom‐up” DL‐based approaches demonstrate their immediate potential for translational applications with their inherent ability to learn complex mappings between diffusion MRI and histology.
DECAM requires multi‐shell dMRI acquisition and incorporates a sophisticated data‐driven DL‐based method to provide neuropathologically accurate measurements. Specifically, high‐resolution multi‐shell dMRI with at least three shells including relatively higher b value (diffusion weighting value, e.g. b = 4500s/mm^2^), regular b value (e.g. b = 1500s/mm^2^), and b = 0 dMRI were used as data inputs to an optimized DL framework. Higher b‐values are physically critical for revealing non‐Gaussian diffusion arising from restricted diffusion in cellular compartments such as somas and neurites, which are prevalent in the cerebral cortex [26, 27, 33]. Since the more restrictive a compartment is, the smaller signal drop can be detected at a lower b value, a larger b‐value is needed to allow the detection of sufficient compartmental signal drops distinguishable from noise. We explicitly filtered out somas with a radius smaller than 5.6 µm to ensure accurate quantification of histology‐derived soma density within the sensitivity range of dMRI, consistent with the literature [47] suggesting that dMRI is only sensitive to somas with a radius larger than 3–7 µm. To provide high‐quality training data to the DL framework, we also addressed the challenge of accurate alignment between MRI and histology in primate brains with complex morphology by employing cortical label vectors using one‐hot encoding. The cortical label vectors directed DECAM to learn exclusively from accurately registered voxels. In addition to dMRI signals containing rich microstructural information as inputs and accurately registered dMRI‐histology pairs as training data, the DL framework was also optimized for generating high‐fidelity cortical architecture maps. We adopted GAN, a highly efficient framework for synthesizing realistic images across modalities. The GAN consists of a generator for estimating target images, i.e., histology images in this study, and a discriminator for ensuring that the generator faithfully estimates ground‐truth histology. To overcome the common vanishing gradient issue that the traditional GAN's generator stops updating, we developed the BRC to explicitly model the coupling between the generator and the discriminator. In BRC‐GAN, the generator's gradient includes a hierarchical coupled‐response gradient term to precisely capture its dependence on the discriminator. The BRC‐GAN maintained strong gradient signals throughout training and enhanced GAN efficacy in estimating cellular architectural details. The prominent DECAM features include the multi‐shell dMRI input, DL framework optimized by BRC, and innovative cortical label vectors. These DECAM features jointly enable accurate and reproducible estimation of cortical cytoarchitecture in primate brains.
The whole‐brain heterogeneous cortical architecture maps estimated by DECAM align well with existing histological findings [42, 43, 44, 48, 49, 50], and such microstructural heterogeneity underlies regionally heterogeneous brain functions. Higher SD can be found in the frontal and occipital lobes than in the parietal and temporal lobes in this study, consistent with histological findings of the SD profile along the anterior to posterior directions of primate brains [42, 43, 44]. The cytoarchitecture characterized by higher SD in the frontal lobe and occipital lobe underlies specific functions. The more sophisticated higher‐order brain functions in the frontal lobe require denser neurons, and the visual cortex includes more than 30% of total cortical neurons [44], hence associated with higher SD, for the heavy visual processing. Besides the high fidelity of estimated whole‐brain SD, the heterogeneous cortical architecture pattern was also consistent across macaque brains, highlighting the inter‐subject reproducibility for the SD profiles estimated by DECAM. The quantitative histology‐based validation and inter‐subject correlation tests ensure DECAM's high fidelity and reproducibility and suggest that the noninvasive DECAM platform serves as a potential substitute for invasive histology to reveal the cytoarchitectural changes underlying brain function changes across the lifespan or in disorders.
The clear outperformance of DECAM‐estimated SD over cortical microstructural measures from widely used dMRI signal models such as NODDI, DTI, and DKI indicates the high accuracy and specificity of DECAM in estimating SD. DECAM is optimized to directly and accurately estimate distinctive features of somas within the cortex, reflected by a higher correlation between DECAM‐estimated SD and ground‐truth SD. Those widely used dMRI signal models incorporate simplified biophysical assumptions and have limited sensitivity to the specific cytoarchitectural features relevant to SD, hence lacking the required specificity to differentiate SD. For example, NODDI [29] has demonstrated extensive strength in modeling neurites in white matter, the cytoarchitectural component distinctive from the somas. Due to the very complicated cytoarchitecture in the cerebral cortex, abundant caution needs to be taken when choosing models for specific cortical cytoarchitectural measures [13, 31]. We demonstrated that cortical DKI‐derived MK is sensitive to the cortical cytoarchitectural changes in brain development [32] and is sensitive to the cortical cytoarchitectural heterogeneity [33]. However, MK cannot distinguish soma contribution from neurite contribution to the cortical microstructural barrier, resulting in low correlation between DKI metric measure and SD. Although the highest b‐value is sufficient for sensitivity to somas [28], without a sufficient number of b values, another dMRI signal model, soma and neurite density imaging (SANDI) [28], more tailored for modeling cortical cytoarchitecture, was not tested. It is expected that SANDI might estimate SD with higher correlation with the ground truth SD, compared to those dMRI signal models included in this study. Comparison with the dMRI signal models suggests the limitations of these signal models in estimating the soma‐specific measures.
Several limitations, technical considerations, and future aspects are noteworthy. Currently, the raw ex‐vivo dMRI images with certain gradient directions and b‐values were used for training. The presented DECAM framework is generalizable in a way to incorporate other multi‐shell dMRI datasets by adding additional channels to the input layer of the GAN generator, given that paired histology images of these other multi‐shell dMRI datasets are also available. Future incorporation of in vivo dMRI data will require careful adoption of advanced preprocessing pipelines for in vivo dMRI as suggested in the Methods section. Although there is no intra‐subject test‐retest data, intra‐subject consistency and reproducibility across scans are reasonable, with rigorous scanner maintenance protocol also specified in the Methods section. When selecting histological images, factors such as histological data staining homogeneity, sub‐micrometer resolution, no damage, and morphology, and similarity to the MRI brain limited the options of matched histological datasets. Based on these criteria, the selection of histological datasets of only 37 coronal slices in this study resulted in each sampled histological slice corresponding to each dMRI plane with 2 mm thickness. Future work should incorporate histological datasets with a larger number of coronal slices, with each dMRI plane corresponding to aggregated multiple histological slices making up the histological slab, with the same thickness as the dMRI plane. In this study, the underlying assumption of matching the dMRI plane and the histological slice with different thicknesses is that the histological slice represents the histological slab with the same thickness as the dMRI plane. Super‐resolution models trained between high‐ and low‐resolution histology slices could be applied to current low‐resolution DECAM outputs as a promising direction to bridge the scale gap between dMRI‐based virtual pathology and actual neuropathology. The macaque brain was used as it is characterized by prominent within‐subject cytoarchitecture variability across gyri [33]. And relatively smaller inter‐subject cytoarchitecture variability for the same gyrus compared to the human brain [51], mitigating bias caused by not using the same brain for histology and MRI. Future studies incorporating training data with high‐resolution dMRI and histology of the same brains, as well as a larger sample size, would yield a more accurate estimation of cortical architecture and enable within‐subject validation. The primary role of cortical label vectors in the current framework is to mitigate residual misalignment between dMRI and histology across different brains, regardless of whether the brains are pathological, normal, or what specific atlas is used. A potential source of bias may come from learning from morphological features. However, since the framework inputs are patch‐based, the cortical label vectors do not contain global morphological features. Although we only demonstrated estimation of SD, given other types of staining, such as neurofilament staining, estimation of other types of cortical architecture, such as neurite density, can also be achieved based on the DECAM framework. DECAM serves as a proof‐of‐concept work for noninvasively mapping cortical architecture using dMRI and DL in primate brains, and should not be interpreted as immediately generalizable to human studies with varying pathological conditions and acquisitions. However, despite a few challenges such as resolution differences between histology and dMRI, scarce matched brain MRI and histology datasets with pathological conditions, and the demand of extensive computational power, in principle a foundation model could be established based on the presented DL‐based framework if a vast amount of paired dMRI and histology dataset in both healthy and pathological subjects across lifespan are available as training datasets. Future studies leveraging brain histology and dMRI data across different developmental stages, including fetal, neonatal, adolescent, and adult stages, from BRAIN Initiative Cell Atlas Network (BICAN, https://www.portal.brain‐bican.org/) can expand DECAM applicability. With future dMRI technology advances, the DECAM framework with dMRI data of much higher resolution (e.g., less than isotropic 0.1 mm) as inputs may enable delineation of cortical architecture from different cortical layers.
Conclusion
4
We developed a cutting‐edge noninvasive DL‐based framework for estimating cortical cytoarchitecture in primates. The results highlight high‐fidelity, reproducible whole‐brain cortical cytoarchitecture estimation using dMRI. Overall, DECAM represents a promising digital medicine technique that can profoundly advance precision and individualized medicine.
Methods
5
High‐Resolution Nissl‐Stained Histology Data and Soma Density Quantification
5.1
A high‐resolution macaque brain Nissl‐stained histology dataset of 55 000 dpi (resolution of 0.46 µm) was downloaded from a publicly available resource [43] and demonstrated in Figure S1. Nissl‐stained coronal histological slices of macaque brain were scanned at 0.46 µm/pixel using an Aperio ScanScope T3 scanner [43]. Nissl‐staining stains the somas of both neurons and glia, the major underlying neuroanatomical cytoarchitectures in the cerebral cortex. Twelve representative coronal slices showing the least distortions compared to dMRI were selected in the middle part of the brain. The coronal slices toward the anterior and posterior of the brain demonstrated relatively larger distortions and separation of the two hemispheres and were excluded from further analysis. We quantified the SD from these 12 histology images, serving as the ground truth in the DL framework. For accurately measuring SD (Figure 3a bottom panel), the Nissl‐stained images were blocked into segments of 1304 × 1304 pixels with each segment size of 0.6 × 0.6 mm^2^, consistent with the in‐plane resolution of ex‐vivo macaque brain dMRI elaborated below. All histology segments were then gray‐scaled and thresholded. SD was defined as the number of contoured units per segment area (# of soma/mm^2^). The Nissl‐stained neuronal and glial cell bodies are both counted. The literature [47] suggests that dMRI is only sensitive to somas with a radius larger than 3–7 µm. A contoured area threshold of 100 µm^2^ corresponding to a radius threshold of 5.6 µm was applied to filter out staining artifacts and ensure accurate quantification of histology‐derived SD within the sensitivity range of dMRI. As illustrated in Figure S2, the calculated high SD matches well with the dense soma appearance in the Nissl‐stained histology image and vice versa. The calculated SD maps are also consistent with the literature [52]. We then registered the histology images along with SD maps to the dMRI space (Figure 3a) to prepare the training data for the DL framework.
High‐Resolution Multi‐Shell dMRI in Ex‐Vivo Macaque Brains and Preprocessing
5.2
Brain samples from three young adult rhesus macaques (aged 4.7, 5, and 5.5 years, respectively; 2 m/1F) were obtained by perfusion fixation with 4% paraformaldehyde after anesthesia with intramuscular injection of ketamine hydrochloride. The procedures on macaques were conducted with great care to ensure the well‐being of the macaques and were approved by the Institutional Animal Care and Use Committee with approval number FMKTY080990 at Johns Hopkins University, where the procedures were performed. High‐resolution multi‐shell dMRI acquisitions were conducted with a 3T Philips Achieva MR system using an eight‐channel knee coil at room temperature. Two b‐values (b = 1500, 4500 s/mm^2^), each with 31 independent diffusion‐weighted directions [53], were used to acquire coronal planes with a single‐shot echo‐planar imaging (EPI) sequence. Sensitivity encoding parallel imaging scheme (SENSE, reduction factor = 2) was applied. The b‐values were optimized such that non‐Gaussian diffusion in the cerebral cortex was captured, and the signal‐to‐noise ratio (SNR) for the higher b‐value dMRI image remains sufficient, as described in our previous publication [33]. High‐resolution dMRI parameters were: field of view (FOV) = 100 × 100 × 72 mm^3^, in‐plane imaging matrix = 166 × 166, in‐plane resolution = 0.6 × 0.6 mm^2^, slice thickness = 2 mm, repetition time/ echo time (TR/TE) = 2100/77.8 ms, and number of scan averages = 24. Two repetitions were performed for each b‐value acquisition to increase the SNR, resulting in a total acquisition time of 17 h for scanning each macaque brain. Accordingly, datasets of four sessions with two sessions per b value, 1500 or 4500 s/mm^2^, were obtained for each macaque brain. Figure S3 demonstrates an example of a high‐resolution macaque brain multi‐shell dMRI dataset. A 12‐parameter affine registration aligning all diffusion‐weighted image volumes to b0 image volume of the same session was conducted. This process was followed by intersession 12‐parameter affine registrations aligning b0 image of the source session to that of the target session and the transformation matrix was applied to all dMRI volumes in the source session. Co‐registered dMRI volumes of a specific b‐value from two repetitions were averaged. This results in 64 averaged dMRI volumes, including 31 dMRI volumes with b = 1500 s/mm^2^, 31 dMRI volumes with b = 4500 s/mm^2,^ and 2 b0 volumes. These dMRI datasets serve as inputs to the DL framework. To enhance intra‐subject consistency and reproducibility across scans, we routinely conduct rigorous quality assurance (QA) and quality control (QC) protocols as follows every day. General MRI slice and slice‐time integral measures of QC are determined daily using ADNI and BIRN phantoms, and significant deflections from normal variation will identify any systematic anomaly, which will be addressed immediately with the vendor technical support and/or the in‐house MR physicist team. As is our common laboratory practice, test‐ reliability of any imaging protocol like the one described above was assessed with a minimum of 4 subjects X 4 repeat estimation on intra‐ and inter‐subject variation. It is noteworthy that unlike in vivo dMRI studies often incorporating preprocessing such as B0 susceptibility artifact correction and denoising [54] to reduce noise and geometric distortion artifacts, in the ex vivo imaging setting for the present study, the need for preprocessing was greatly reduced because lengthy acquisitions of 24 repetitions increased SNR, skull removal improved the precision of brain boundaries, and immersion in Fomblin minimized susceptibility artifacts.
Registration of dMRI and Histological Images
5.3
To transfer the SD maps quantified from histology to dMRI space (Figure 3a), Nissl‐stained histology images were gray‐scaled to drive the registration between histology and dMRI. Nissl‐stained histology and dMRI in visually corresponding coronal slices were selected for registration at the 2D slice level. Gray‐scaled Nissl‐stained images were registered to the average diffusion‐weighted image of a representative macaque subject using a 12‐parameter affine transformation, followed by a non‐linear LDDMM transformation [40] capable of elastic warping. LDDMM was applied using DiffeoMap software (mristudio.org). The combined affine and LDDMM transformation resulted in a joint histology‐dMRI space. The SD maps calculated in the histology native space were registered to the joint dMRI‐histology space using the same affine and LDDMM transformation matrices. The co‐registered SD maps from histology and dMRI volumes were then segmented into 3 × 3 patches, serving as training data.
DECAM Deep Learning Framework
5.4
The computational architecture of DECAM for estimation of whole‐brain SD in macaque brains is outlined in Figure 3, and implementation details can be found in Methods. DECAM incorporates a state‐of‐the‐art BRC [38] for optimizing a deep learning framework, as well as novel cortical label vectors (Figure 4; see details in the “Creation of cortical label vectors for overcoming histology‐MRI misregistration” section below) for resolving the challenge of cross‐modality misregistration. The registered SD maps, dMRI volumes, along with the cortical label vectors, all segmented into 3 × 3 patches, were used for training, validation, and test sets. Specifically, the leave‐one‐out method was used to reserve one slice at a time as the test set, and the rest of the slices were used in training and hyperparameter tuning with 1000 randomly selected patches held out as a validation set (Figure 3b; Figure S9). The trained DL network was then applied to the test slice to estimate SD maps (Figure 3c), and to the other slices of dMRI data from the macaque brain used for training, as well as all slices from two other macaque brains (Figure S8).
BRC‐GAN Framework
5.5
The BRC‐GAN framework consists of a generator G, a discriminator D, and a BRC. The generator (green box in Figure 3b) takes patch‐based inputs Z of size 3 by 3 by number of channels. It consists of 30 ResBlocks [53]. The first ResBlock has 130 input channels (i.e., 64 dMRI channels and 66 cortical label vector channels reflecting cortical regions in a digital macaque atlas ([41]) and 64 output channels, while all the other ResBlocks have 64 input and 64 output channels. The ResBlocks are connected to a final convolutional layer (Conv) that outputs G(Z), the estimated SD. The discriminator (red box in Figure 3b) consists of three Conv‐leaky rectified linear unit (LeakyReLU)‐batch normalization (BN) combinations followed by another Conv layer and a sigmoid layer (Figure 3b), and outputs D(G(Z)), the discriminator's judgement on whether G(Z) is estimated or ground‐truth.
There are two loss functions involved in training the GAN. The generator loss function consisting of an *L_2_
θ G and θ D are the generator and discriminator parameters, respectively. X represents the 3 × 3 ground‐truth SD image patch. F(θ G, θ D(θ G)) embeds the dependency of the generator loss function on the discriminator parameters using θ D(θ G), and is reparameterized as ϕ(θ_G_).
The discriminator loss function, consisting of two adversarial loss terms and a gradient penalty term [36, 51] can be written as:
To implement BRC, the dependency of generator parameters θ G on discriminator parameters θ D is modeled through adding ∂F(θG,θD)∂θD×∂θD(θG)∂θG to the gradient of the generator loss function ∂F(θG,θD)∂θG, and can be written as:
The best response gradient ∂θD(θG)∂θG, explicitly formulating the dependency of the generator on the discriminator, can be well approximated by [38]:
For implementing the BRC, the pseudo‐code is listed below:
Input: Initialization of θ_ G _ and θ_D_ and necessary parameters
Output: The optimal generator G
- 1: while not converging do
- 2: calculate the gradients ∇θDf=∂f∂θD, ∇θDF=∂F∂θD
- 3: compute best response constraint (BRC) loss:
- 4: Update the generator parameters:
- 5: Update the discriminator parameters:
- 6: end while
- 7: Return the optimal parameters θ_G_
DECAM Implementations
5.6
DECAM was implemented with PyTorch framework and trained on NVIDIA GeForce RTX 3090 GPU with 24 GB memory. All DL networks were trained for 40 epochs with a batch size of 128, using the adaptive moment estimation (Adam) optimizer for network optimization. The initial learning rates were all set to 2 × 10^−4^, and a learning rate scheduler (StepLR) was used with gamma = 0.1 and step size = 10. The DL parameters at the epoch that yielded the minimal validation loss were selected.
Creation of Cortical Label Vectors for Overcoming Histology‐dMRI Misregistration
5.7
Registration between histology and MRI was described in the section in “Registration of dMRI and histological data”. Significant misregistration occurs across the cortical regions, as demonstrated in Figure 4a as an example. To resolve the histology‐MRI registration bias, we created novel cortical label vectors and fed them into the DL network. Briefly, we mapped 66 cortical labels defined from a digital macaque atlas [41] into the dMRI space, which histology slices were also registered to, by transferring the parcellated cortical labels from the macaque atlas [41]. The contrast of the averaged diffusion weighted image was used to drive the affine and LDDMM registrations for cortical label mapping. The cortical labels were further manually refined afterward in both the dMRI and histology images. We extracted the overlap area of the cortical region with a specific label in the co‐registered dMRI‐histology space and considered these voxels in the overlap areas as accurately registered voxels of a given cortical label. In this way, despite residual misregistration, it is ensured that the input data for the DL network are exclusively from histology‐dMRI voxels matched at the same gyrus, which represent almost the same cellular architecture, as demonstrated in, e.g., Figure S2. We one‐hot encoded [55] cortical label vectors associated with the input dMRI data and fed them into the DL framework in patches as described in the above “BRC‐GAN framework” section. The data from histology voxels matched at the same gyrus was used for the ground truth in the DECAM training.
Evaluation of DECAM Performance Compared to Other DL Frameworks
5.8
To provide a comparative assessment of the DECAM DL framework performance (BRC‐GAN with cortical label vectors), we estimated cortical SD maps using the traditional GAN [37], and GAN with cortical label vectors. All frameworks were trained with the same batch size, type of optimizer, and learning rates. The framework parameters at the epoch that yielded the minimal validation loss were chosen for the testing phase. To evaluate the performance of a DL framework, Pearson correlation and residual were computed between the ground‐truth SD and estimated measures in cortical regions. In general, higher values of the Pearson correlation coefficient and lower values of the residual indicate better performance of the DL frameworks. We compared the mean correlation coefficients and mean residual values from representative test slices across these three frameworks by first applying a repeated‐measures one‐way analysis of variance (ANOVA) to assess the overall effect of the framework on these performance metrics. Following the significant ANOVA findings, post‐hoc paired t‐tests were employed to conduct pairwise comparisons between frameworks, with FDR correction applied to account for multiple comparisons. Distributions of voxel‐wise difference between DECAM‐estimated SD and ground‐truth SD were computed for BRC‐GAN with cortical label vectors, BRC‐GAN without cortical label vectors, GAN with cortical label vectors, and GAN. Additionally, we evaluated the performance of two competitive patch‐based DL frameworks, ResNet [56] and UNet [57], in estimating cortical SD, and compared them with a traditional GAN framework. All training settings were kept consistent across these three frameworks, except that UNet was trained using a minimal image patch size of 8 × 8. Detailed comparison results are shown in Figure S4.
Evaluation of Fidelity and Reproducibility of DECAM Estimation
5.9
To assess the fidelity of DECAM estimation against ground‐truth histology, we used the leave‐one‐out method (Figure S9) and conducted Pearson correlation between the estimated SD and the ground‐truth SD (r observed) at both individual slice and cortical gyral levels. For the slice‐level correlation, we correlated voxel‐wise values between the estimated and ground‐truth SD within a slice. To further evaluate the fidelity of DECAM estimation at the cortical gyral level, we computed the average SD for each parcellated gyri defined by the digital macaque atlas [41] within the frontal, temporal, parietal, and occipital lobes across all twelve validation slices. We then correlated these average SD values with the corresponding average ground‐truth gyral SD values across all cortical gyri examined. To assess the statistical significance of the observed correlations, we employed a nonparametric permutation test with 10 000 iterations. For the slice‐level analysis, in each iteration, we randomly shuffled the estimated cortical SD values across all voxels within each slice and recalculated the Pearson correlation. Similarly, for the cortical gyral‐level analysis, we randomly shuffled the estimated average SD values across all gyri and recalculated the correlation. This iterative shuffling process generated a null distribution of 10 000 correlation coefficients (r null) for each level. The empirical permutation p value (p perm) was then determined as the proportion of these 10 000 null correlations (r null) that were greater than our observed correlation coefficient between the estimated and ground‐truth SD (r observed). A p perm value below 0.05 was considered statistically significant. Table S1 lists all the presented cortical gyri. This multi‐level validation approach ensures that DECAM‐derived SD maps reliably capture spatial variations in cellular density with high consistency across anatomical regions.
To assess the reproducibility of DECAM estimation, we applied the trained framework on two held‐out macaque brains that were never used in training and estimated their whole‐brain SD maps. The whole‐brain SD map for the macaque brain used in training was also estimated. To further evaluate cross‐subject reproducibility, SD values were extracted along the cortical ribbon within a predefined slab region. The left and right normalized cortical SD profiles were analyzed for inter‐subject consistency. To quantify SD reproducibility at the regional level, SD values were averaged within each parcellated cortical gyrus defined by the digital macaque atlas [41] across three macaque brains, and pairwise Pearson correlations of gyral SD were computed. Similarly, a nonparametric permutation test with 10 000 iterations was used to determine the statistical significance of each correlation.
Comparison Between DECAM and Widely Used dMRI Signal Models for Estimating Cortical Cytoarchitecture
5.10
To assess the performance of DECAM in estimating cortical cytoarchitecture compared to widely used dMRI signal models, we fitted DTI, DKI, and NODDI to the representative macaque brain, which was registered with histology. Diffusion tensor fitting was conducted with dMRI of b 1500 s/mm^2,^ besides b0, by using DTIStudio [58]. The voxel‐wise diffusion kurtosis was fitted using in‐house MATLAB code (github.com/ritaz0904/DK_fitting) with constrained linear fitting [59]. The NODDI Matlab toolbox (https://www.nitrc.org/projects/noddi_toolbox/) was used to generate the intra‐neurite volume fraction map. Cortical voxels in DTI‐derived FA, MD, DKI‐derived MK, NODDI‐derived intra‐cellular volume fraction map, and DECAM‐estimated SD were extracted from the registered slices between dMRI and ground‐truth histology. Pearson correlations with permutation test between dMRI signal model metric measures or DECAM‐estimated SD and ground‐truth SD were computed. The same permutation testing elaborated in the above section “Evaluation of fidelity, and reproducibility of DECAM estimation” was applied. Density plots with a linear fitted line were generated for each metric measure. A line plot for comparing Pearson correlation coefficients between ground‐truth SD and dMRI signal model metric measures, as well as DECAM‐estimated SD, was generated.
To provide an overview of estimated metric measure variations across the brain, ground‐truth SD, DECAM‐estimated SD, and dMRI signal model metric measures were mapped onto the cortical surface. To reveal metric measure variations across cortical regions and facilitate comparisons with ground‐truth SD, we extracted normalized ground‐truth SD, DECAM‐estimated SD, and dMRI signal model metric measures across cortical gyri and plotted them as violin plots. The cortical gyri of a representative slice were parcellated with the digital macaque atlas [41].
To evaluate the robustness of DECAM across individual subjects used for training and further compare its performance to dMRI signal models, we trained three independent DECAM model versions (DECAM‐1, DECAM‐2, and DECAM‐3) on macaque brains #1, #2, and #3 with the same matched histology, respectively, and validated them on all three macaque brains. For each model version, voxel‐wise Pearson correlations between estimated SD and ground‐truth SD were calculated in the same manner as described above, and compared with those obtained for FA, MD, MK, and NODDI‐derived ICVF. Boxplots were generated to summarize the distribution of correlation coefficients across slices, enabling direct comparison of cross‐subject performance between DECAM model versions and dMRI signal models.
Statistical Analysis
5.11
Statistical analyses were conducted in R4.5.0 using custom code, with *p < 0.05 set as the significance threshold. To assess the statistical significance of the observed Pearson's correlations, we employed a nonparametric permutation test. One‐way ANOVA was used for comparison between four groups, followed by post‐hoc paired t‐tests with FDR correction. Detailed descriptions of the statistical tests and sample sizes for each experiment are provided in the figure legends and main text.
Author Contributions
H.H. designed the study. T.Z., M.O., and H.H. designed the deep‐learning network. X.L. and R.L. contributed to the optimization of the deep‐learning network. H.H. and M.O. coordinated and oversaw the study. T.Z., M.O., S.T., J.G, Z.Z., and H.H. generated figures. All authors participated in the discussion and interpretation of the data. T.Z., M.O., and H.H. wrote the manuscript.
Conflicts of Interest
The authors declare no conflicts of interest.
Code Availability
Source code used in analysis and related documentations are also available at the Huang lab GitHub repository (https://www.github.com/haohuanglab/DECAM).
Supporting information
Supporting File: advs73562‐sup‐0001‐SuppMat.docx.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1P. R. Huttenlocher and A. S. Dabholkar , “Regional Differences in Synaptogenesis in Human Cerebral Cortex,” The Journal of Comparative Neurology 387 (1997): 167–178.9336221 10.1002/(sici)1096-9861(19971020)387:2<167::aid-cne 1>3.0.co;2-z · doi ↗ · pubmed ↗
- 2J. C. Silbereis , S. Pochareddy , Y. Zhu , M. Li , and N. Sestan , “The Cellular and Molecular Landscapes of the Developing Human Central Nervous System,” Neuron 89 (2016): 248–268.26796689 10.1016/j.neuron.2015.12.008PMC 4959909 · doi ↗ · pubmed ↗
- 3N. J. Minshew and D. L. Williams , “The New Neurobiology of Autism: Cortex, Connectivity, and Neuronal Organization,” Archives of Neurology 64 (2007): 945–950.17620483 10.1001/archneur.64.7.945PMC 2597785 · doi ↗ · pubmed ↗
- 4P. R. Huttenlocher , “Synaptic Density in Human Frontal Cortex—Developmental Changes and Effects of Aging,” Brain Research 163 (1979): 195–205.427544 10.1016/0006-8993(79)90349-4 · doi ↗ · pubmed ↗
- 5M. P. Forrest , E. Parnell , and P. Penzes , “Dendritic Structural Plasticity and Neuropsychiatric Disease,” Nature Reviews Neuroscience 19 (2018): 215–234.29545546 10.1038/nrn.2018.16PMC 6442683 · doi ↗ · pubmed ↗
- 6P. Penzes , M. E. Cahill , K. A. Jones , J.‐E. Van Leeuwen , and K. M. Woolfrey , “Dendritic Spine Pathology in Neuropsychiatric Disorders,” Nature Neuroscience 14 (2011): 285–293.21346746 10.1038/nn.2741 PMC 3530413 · doi ↗ · pubmed ↗
- 7T. Gómez‐Isla , R. Hollister , H. West , et al., “Neuronal Loss Correlates With but Exceeds Neurofibrillary Tangles in Alzheimer's Disease,” Annals of Neurology 41 (1997): 17–24.9005861 10.1002/ana.410410106 · doi ↗ · pubmed ↗
- 8M. Knobloch and I. M. Mansuy , “Dendritic Spine Loss and Synaptic Alterations in Alzheimer's Disease,” Molecular Neurobiology 37 (2008): 73–82.18438727 10.1007/s 12035-008-8018-z · doi ↗ · pubmed ↗