Model-based Computed Tomography Image Estimation: Partitioning Approach

TL;DR

This paper introduces a partitioning approach using skew Gaussian and Gaussian mixture models to improve CT image estimation from MR images, focusing on enhanced bone tissue accuracy for MR-based radiotherapy.

Contribution

It proposes novel mixture models that partition data into tissue types, significantly improving bone tissue estimation over existing model-based methods.

Findings

Outperforms existing methods in dense bone tissue estimation

Uses partitioning to improve CT estimation from MR images

Validated with leave-one-out cross-validation on real data

Abstract

There is a growing interest to get a fully MR based radiotherapy. The most important development needed is to obtain improved bone tissue estimation. The existing model-based methods perform poorly on bone tissues. This paper was aimed at obtaining improved bone tissue estimation. Skew Gaussian mixture model and Gaussian mixture model were proposed to investigate CT image estimation from MR images by partitioning the data into two major tissue types. The performance of the proposed models was evaluated using leave-one-out cross-validation method on real data. In comparison with the existing model-based approaches, the model-based partitioning approach outperformed in bone tissue estimation, especially in dense bone tissue estimation.

Figures

Equations

$$f\left({\mathbf{y}}_i|\mathbf{\eta },\mathbf{\Sigma },\mathbf{\lambda }\right)=2\mathcal{N}\left({\mathbf{y}}_i|\mathbf{\eta },\mathbf{\Sigma }\right)\Phi \left({\mathbf{\lambda }}^{\prime }{\mathbf{\Sigma }}^{-1/2}\left({\mathbf{y}}_i-\mathbf{\eta }\right)\right),$$

$${\mathbf{Y}}_i=\mathbf{\eta }+{\mathbf{\Sigma }}^{\frac{1}{2}}\mathbf{\delta }{U}_i+{\mathbf{\Sigma }}^{\frac{1}{2}}{\left(1-\mathbf{\delta }{\mathbf{\delta }}^{\prime }\right)}^{\frac{1}{2}}{\mathbf{V}}_i,$$

$$U_i\sim \mathcal{H}\mathcal{N}\left(u_i|0,1,\left(0,\infty \right)\right),{\mathbf{V}}_i\sim \mathcal{N}\left({\mathbf{v}}_i|0,1\right),\text{and}\mathbf{\delta }=\frac{\mathbf{\lambda }}{\sqrt{1+{\mathbf{\lambda }}^{\prime }\mathbf{\lambda }}}.$$

$$\begin{array}{rl}{\mathbf{Y}}_i|U_i=u_i& \sim \mathcal{N}\left({\mathbf{y}}_i|\mathbf{\eta }+u_i\mathbf{\xi },\mathbf{\Omega }\right),\\ U_i& \sim \mathcal{H}\mathcal{N}\left(u_i|0,1,\left(0,\infty \right)\right),\end{array}$$

$$\mathbf{\delta }=\frac{\mathbf{\lambda }}{\sqrt{1+{\mathbf{\lambda }}^{\prime }\mathbf{\lambda }}},\mathbf{\xi }={\mathbf{\Sigma }}^{\frac{1}{2}}\mathbf{\delta },\text{and}\mathbf{\Omega }={\mathbf{\Sigma }}^{\frac{1}{2}}\left(1-\mathbf{\delta }{\mathbf{\delta }}^{\prime }\right){\mathbf{\Sigma }}^{\frac{1}{2}}=\mathbf{\Sigma }-\mathbf{\xi }{\mathbf{\xi }}^{\prime }.$$

$$Z_{ik}=1_{\left(Z_i=s\right)}=\{\begin{array}{ll}1,& \text{if }k=s,\\ 0,& \text{otherwise},\end{array}$$

$$f\left({\mathbf{y}}_i|\mathbf{\Psi }\right)=\sum _{k=1}^K{\pi }_k\mathcal{S}\mathcal{N}\left({\mathbf{y}}_i|{\mathbf{\eta }}_k,{\mathbf{\Sigma }}_k,{\mathbf{\lambda }}_k\right),$$

$${\pi }_k\ge 0,\sum _{k=1}^K{\pi }_k=1,{\mathbf{\psi }}_k=\left\{{\pi }_k,{\mathbf{\eta }}_k,{\mathbf{\Sigma }}_k,{\mathbf{\lambda }}_k\right\},\mathbf{\Psi }=\left\{{\mathbf{\psi }}_1,{\mathbf{\psi }}_2,\cdots ,{\mathbf{\psi }}_K\right\},k=1,2,\cdots ,K.$$

$$\log f\left(\mathbf{y}|\mathbf{\Psi }\right)=\sum _{i=1}^n\log \left\{\sum _{k=1}^K{\pi }_k\mathcal{S}\mathcal{N}\left({\mathbf{y}}_i|{\mathbf{\eta }}_k,{\mathbf{\Sigma }}_k,{\mathbf{\lambda }}_k\right)\right\}.$$

$$\begin{array}{rl}{\mathbf{Y}}_i|U_i=u_i,Z_{ik}=1& \sim \mathcal{N}\left({\mathbf{y}}_i|{\mathbf{\eta }}_k+u_i{\mathbf{\xi }}_k,{\mathbf{\Omega }}_k\right),\\ U_i|Z_{ik}=1& \sim \mathcal{H}\mathcal{N}\left(u_i|0,1,\left(0,\infty \right)\right),\end{array}$$

$${\mathbf{\delta }}_k=\frac{{\mathbf{\lambda }}_k}{\sqrt{1+{\mathbf{\lambda }}_k^{\prime }{\mathbf{\lambda }}_k}},{\mathbf{\xi }}_k={\mathbf{\Sigma }}_k^{\frac{1}{2}}{\mathbf{\delta }}_k,\text{and}{\mathbf{\Omega }}_k={\mathbf{\Sigma }}_k-{\mathbf{\xi }}_k{\mathbf{\xi }}_k^{\prime }.$$

$$\displaystyle\begin{split}\log f\left(\mathbf{y},\mathbf{u},\mathbf{z}|\boldsymbol{\Theta}\right)&=&\sum_{i=1}^{n}\sum_{k=1}^{K}z_{ik}\left\{-\frac{1}{2}\left(\mathbf{y}_{i}-\boldsymbol{\eta}_{k}-\boldsymbol{\xi}_{k}{u}_{i}\right)^{\prime}\boldsymbol{\Omega}_{k}^{-1}\left(\mathbf{y}_{i}-\boldsymbol{\eta}_{k}-\boldsymbol{\xi}_{k}{u}_{i}\right)\right\}+\\ &&\sum_{i=1}^{n}\sum_{k=1}^{K}z_{ik}\left\{\log\pi_{k}-\frac{1}{2}\log|\boldsymbol{\Omega}_{k}|+const\right\},\end{split}$$

$${\mathbf{\theta }}_k=\left\{{\pi }_k,{\mathbf{\eta }}_k,{\mathbf{\xi }}_k,{\mathbf{\Omega }}_k\right\},\mathbf{\Theta }=\left\{{\mathbf{\theta }}_1,{\mathbf{\theta }}_2,\cdots ,{\mathbf{\theta }}_K\right\},k=1,2,\cdots ,K,$$

$$\mathcal{Q}\left(\mathbf{\Theta },{\mathbf{\Theta }}^{\text{old}}\right)=E\left[\log f\left(\mathbf{Y},\mathbf{U},\mathbf{Z}|\mathbf{\Theta }\right)|\mathbf{Y},{\mathbf{\Theta }}^{\text{old}}\right].$$

$$E\left[Z_{ik}|\mathbf{Y},{\mathbf{\Theta }}^{\text{old}}\right],E\left[Z_{ik}{U}_i|\mathbf{Y},{\mathbf{\Theta }}^{\text{old}}\right],\text{and}E\left[Z_{ik}{U}_i^2|\mathbf{Y},{\mathbf{\Theta }}^{\text{old}}\right].$$

$$E\left[Z_{ik}|\mathbf{Y},{\mathbf{\Theta }}^{\text{old}}\right]$$

$${\vartheta }_{ik}$$

$${\psi }_{ik}={\gamma }_{ik}^{\text{old}}E\left[U_i^2|\mathbf{Y},Z_{ik}=1,{\mathbf{\Theta }}^{\text{old}}\right].$$

$$U_i|{\mathbf{Y}}_i={\mathbf{y}}_i\sim \mathcal{T}\mathcal{N}\left(u_i|\frac{{\mathbf{\xi }}^{\prime }{\mathbf{\Omega }}^{-1}\left({\mathbf{y}}_i-\mathbf{\eta }\right)}{1+{\mathbf{\xi }}^{\prime }{\mathbf{\Omega }}^{-1}\mathbf{\xi }},\frac{1}{1+{\mathbf{\xi }}^{\prime }{\mathbf{\Omega }}^{-1}\mathbf{\xi }},\left(0,\infty \right)\right),$$

$$E\left[U_i|{\mathbf{Y}}_i={\mathbf{y}}_i,U_i>0\right]$$

$$var\left[U_i|{\mathbf{Y}}_i={\mathbf{y}}_i,U_i>0\right]$$

$$E\left[U_i^2|{\mathbf{Y}}_i={\mathbf{y}}_i,U_i>0\right]$$

$$\alpha =\frac{{\mathbf{\xi }}^{\prime }{\mathbf{\Omega }}^{-1}\left({\mathbf{y}}_i-\mathbf{\eta }\right)}{\sqrt{1+{\mathbf{\xi }}^{\prime }{\mathbf{\Omega }}^{-1}\mathbf{\xi }}},\beta =\sqrt{1+{\mathbf{\xi }}^{\prime }{\mathbf{\Omega }}^{-1}\mathbf{\xi }},$$

$$E\left[U_i|\mathbf{Y},Z_{ik}=1,{\mathbf{\Theta }}^{\text{old}}\right]\text{and}E\left[U_i^2|\mathbf{Y},Z_{ik}=1,{\mathbf{\Theta }}^{\text{old}}\right],$$

$${\mathbf{\Theta }}^{\text{new}}=\arg \underset{\mathbf{\Theta }}{\max }\mathcal{Q}\left(\mathbf{\Theta },{\mathbf{\Theta }}^{\text{old}}\right),$$

$${\gamma }_{ik}^{\left(m\right)}$$

$${\vartheta }_{ik}^{\left(m\right)}$$

$${\psi }_{ik}^{\left(m\right)}$$

$${\pi }_k^{\left(m+1\right)}$$

$${\mathbf{\eta }}_k^{\left(m+1\right)}$$

$${\mathbf{\xi }}_k^{\left(m+1\right)}$$

$${\mathbf{\Omega }}_k^{\left(m+1\right)}$$

$${\mathbf{\Sigma }}_k^{\left(m\right)}$$

$${\mathbf{\lambda }}_k^{\left(m\right)}$$

Full text

1 Introduction

Magnetic resonance (MR) imaging and computed tomography (CT) are the most widely used diagnostic imaging technologies in medicine. They are used to obtain more detailed cross-sectional images of human body. CT uses ionizing radiation to record a pattern of radiodensities to obtain cross-sectional images. The ionizing radiations are attenuated as they pass through the tissues of patients. The amount of attenuations depends on the tissue types. The differences in attenuation between adjacent tissues create contrast on CT images. Tissues with higher (or lower) attenuation appear brighter (or darker) on grayscale CT images. As a result, air, soft, and bone tissues appear as darkest, darker, and white on grayscale CT images. Therefore, CT image is excellent for identifying and assessing the structures of bone tissues. On the other hand, exposing a patient to ionizing radiation in CT imaging may have a risk for radiation-related cancer.

MR imaging is remarkably different from CT. It does not depend on ionizing radiations. MR imaging relies on the absorption and emission of radio waves from tissue protons exposed to a strong magnetic field. Thus, MR imaging is safer than CT imaging. The relative MR signal intensity differences between adjacent anatomic structures determine tissue contrast on MR images. In comparison with CT images, MR images show much better soft tissue contrast and noticeably improves the delineation of tumors. However, MR images are poor in depicting bone tissues. The reason is that bone, air, and rapidly flowing blood appear black on grayscale MR images.

A new innovation MR-only based radiotherapy enhances tissue contouring and precision in soft tissue therapy setup. It also improves biological information at treatment planning and avoids registration errors, which are errors due to transformation of different images of the same scene into one coordinate system, between CT and MR images. Moreover, MR-only based radiotherapy is a cost effective approach as it reduces redundant imaging. However, co-registered CT and MR image complement each other due to better bone tissue imaging of CT [12, 4, 6]. MR images are not directly applicable for attenuation correction. Attenuation refers to the loss of signals due to absorption or scattering out of the signals in the body. CT images are vital for attenuation correction in positron emission tomography (PET) imaging. This is due to the direct relation between CT image intensities and PET attenuation coefficients. However, CT scanner is not available in recently combined PET/MR imaging scanner. Therefore, MR-only based radiotherapy and the combined PET/MR imaging scanner can be successful if one can obtain a reliable CT image estimation. As a result, we need to develop a reliable CT image estimation method from MR images.

Huynh et al. [9] used a learning-based method to estimate CT image from MR image. A patch of CT image is estimated directly from a given MR image patch using structured random forest. The robustness of the estimation has been evaluated using a new ensemble model. Nie et al. [18] proposed a 3D deep learning-based method for patch-wise estimation of CT images from MR images. The neural network generates structured output and it preserves the neighborhood information in the estimated CT image. Arabi et al. [1] suggested a two-step atlas-based algorithm to estimate CT image from MR image sequences. The estimation is mainly concerned with pinpointing of bone tissues.

Johansson et al. [10] used a Gaussian mixture model (GMM) to obtain CT substitute from MR images without taking spatial dependence between neighboring voxels into account. Johansson et al. [11] investigated the uncertainty associated to the voxel-wise estimation of CT images. By considering spatial dependence between neighboring voxels, Kuljus et al. [14] extended the work of Johansson et al. [10] by using hidden Markov model (HMM) and Markov random field model (MRF). Kuljus et al. [14] compared the estimation quality of GMM, HMM, and MRF. In terms of mean absolute error, HMM outperformed the other models and it was computationally robust than MRF. However, it had a weaker estimation quality on dense bone tissues. Even though MRF had superior performance on bone tissue estimation, it was computationally expensive.

The main aim of this article is to further investigate the voxel-wise estimation of CT images from MR images by partitioning the data into non-bone and bone tissues. According to Johansson et al. [11] and Kuljus et al. [14], the estimation of CT image was poor on air and bone tissues. It is this result that motivated the partitioning of the data into non-bone and bone tissues in order to further explore the estimation of CT images. Even though there is no clear-cut CT image intensity boundary between these tissue types, Waterstram-Rich and Gilmore [21] and Washington and Leaver [20] provide informative threshold delimiting these tissues. Waterstram-Rich and Gilmore [21] used 150 Hounsfield units (HU) as a lower limit for bone tissues. On the other hand, Washington and Leaver [20] utilised 200HU as an approximate delimiting value of the tissues.

The partitioning of the data may introduce skewness. Consequently, there is a need to relax the normality assumption used in [10, 11, 14]. Azzalini [2] proposed a univariate skew-normal model that relaxed the normality assumption by incorporating a skewness parameter in the distributional assumption. Azzalini and Dalla Valle [3] extended the univariate skew-normal to a multivariate skew-normal. A multivariate skew-normal is a tractable extension of a multivariate normal distribution with extra parameter to regulate skewness. Lin et al. [17] introduced a univariate skew-normal mixture model in order to deal with population heterogeneity and skewness. Lin [16] extended the univariate skew-normal mixture model to a multivariate skew-normal mixture model which is an alternative to the most widely used multivariate Gaussian mixture model.

Kahrari et al. [22] developed a multivariate skew-normal-Cauchy distribution and represented it as a shape mixture of the multivariate skew-normal distribution. Kahrari et al. [23] modified the multivariate skew-normal-Cauchy distribution and the modified version becomes a shape mixture of a special case of the fundamental skew-normal distribution developed by Arellano-Valle and Genton [24] with a univariate half-normal mixing distribution. The class of scale mixtures of skew-normal-Cauchy distributions has been represented as a shape mixture of the class of scale mixtures of skew-normal distributions with a univariate half-normal mixing distribution [23]. Jamalizadeh and Lin [25] presented the scale-shape mixtures of skew-normal distributions for modeling asymmetric data. Arellano-Valle et al. [29] established a flexible class of multivariate distributions obtained by both scale and shape mixtures of multivariate skew-normal distributions. Based on the skew-t-normal distribution [27], Tamandi et al. [26] introduced the shape mixtures of the skew-t-normal distributions that contain one additional shape parameter to regulate skewness and kurtosis. The shape mixtures of the skew-t-normal distributions are a flexible extension of the skew-t-normal distribution. Lin et al. [28] developed a multivariate extension of the skew-t-normal distribution that is obtained as a scale mixture of the multivariate skew-normal distribution introduced by Azzalini and Dalla Valle [3]. Based on the multivariate skew-t-normal distribution, Lin et al. [28] introduced a robust probabilistic mixture model which is composed of a weighted sum of a finite number of different multivariate skew-t-normal densities. The flexible mixture model based on the multivariate skew-t-normal distribution includes mixtures of normal, t and skew-normal distributions as special cases. Cabral et al. [5] proposed mixture models which consist of members of skew-normal independent distributions (the skew-normal, the skew-t, the skew-slash and the skew-contaminated normal) and the mixture models are developed using the multivariate skew-normal distribution in [3].

The most common approach to estimate the parameters of these skew-mixture models is the EM algorithm [7]. However, its M-step for the recent skew-mixture models is computationally intractable. Alternatively, we use an EM-type algorithm to estimate these skew-mixture models. That is, we make further assumptions at the M-step of the EM algorithm. For instance, expectation conditional maximization (ECM) algorithm [30], which replaces the M-step of EM with several computationally simple conditional maximization steps, can be enough to estimate the parameters of some mixture models. In mixtures of multivariate skew-t-normal distributions and shape mixtures of skew-t-normal distributions, expectation conditional maximization either (ECME) algorithm [31] was exploited to estimate its parameters by replacing some conditional maximization-steps of ECM with the conditional maximization likelihood step that maximizes the correspondingly constrained actual-likelihood function. Cabral et al. [5] developed an EM-type algorithm that removed some obstacles (for instance, Monte Carlo integration) during parameter estimation process in mixture models of skew-normal independent distributions. In this article, we used a mixture model based on the multivariate skew-normal distribution in [3] and developed EM-algorithm for its parameter estimation. That is, further assumption is not required at M-step of EM algorithm to estimate the parameters of the skew-Gaussian mixture model.

In this work, we use skew-Gaussian mixture model (SGMM) and Gaussian mixture model (GMM) to further explore the estimation of CT images by partitioning the data into two major tissue types: non-bone and bone tissues. Non-bone tissues consist of subclasses such as white matter, blood, water, fat, gray matter, air, etc while bone tissues consist of subclasses such as cranium, mandible, frontal bone, nasal bone, orbital bones, cortical bone, cancellous bone etc. These facts motivated us to apply mixture models on these tissue types. SGMM involves a weighted sum of the joint skew-normal distributions of a CT image intensity and its corresponding intensities of MR images. The number of skew-normal distributions in the mixture depends on the number of underlying tissue types or clusters. Latent variables that represent the underlying tissue types are utilized during parameter estimation process of the model through incomplete data assumption in EM-algorithm framework [7]. Voxel-wise point estimator of CT image was obtained as a weighted sum of the conditional expected value of a CT image intensity given its corresponding intensities of MR images and the underlying tissue type. The probability that an underlying tissue type is determined based on the intensities of MR images was used as a weight of the conditional expected value.

In summary, this study is concerned with comparing the CT image estimation performance of the partitioning approach with HMM, MRF, and GMM on bone tissues. The models HMM , MRF, and GMM are trained on the full data (data that are not partitioned into non-bone and bone tissues). We are also interested to compare the predictive quality of the partitioning approaches, SGMM and GMM* (GMM applied to each partition), on bone tissues.

This article is organized as follows. The second section describes data acquisition and demonstrates statistical methods. The third section presents the results obtained and the final section discusses the implication of the results.

2 Statistical methodology

In this section, we describe the data, formulate SGMM, and develop parameter estimation method. We also demonstrate CT image estimation method and present evaluation method of the estimation. For the remaining models (GMM, HMM and MRF), we refer to Kuljus et al. [14] and the references therein.

2.1 Data acquisition

CT and MR images were obtained from head of five patients. Four MR images were acquired from each patient using two dual echo ultrashort echo-time sequences with flip angles of 10 degrees and 30 degrees. The ultrashort echo-time sequences sampled a first echo (free induction decay) and a second echo (gradient echo) from the same excitation with an echo time of 0.07 and 3.76 milliseconds. MR image of a patient was reconstructed to $192\times 192\times 192$ matrix. An entry in the matrix represents a signal intensity corresponding to a three dimensional tissue (voxel) with size 1.33 $\times$1.33 $\times$1.33 mm3. One CT image of a patient was acquired using gradient echo Lightspeed with 2.5mm slice thickness. The acquired CT image was reconstructed with an in-plane resolution of 0.78 $\times$ 0.78 mm2. One binary mask (an image with voxel value 1 (or 0) representing the region of interest (or the surrounding air)) was also developed to demarcate the head of a patient from its surrounding air. The main use of the binary mask is to exclude the surrounding air from the acquired CT and MR images. For each patient, the binary mask, the CT image, and the four MR images were co-registered and resampled to the same resolution (voxel-to-voxel correspondence and set to the same voxel dimension) using linear interpolation. For further technical details, we refer to Johansson et al. [10]. Voxel values of the CT image, the binary mask, and the four MR images were organised into six columns to obtain data for a patient. The organised data of each patient were column stacked and the surrounding air removed to obtain data for model fitting. Figure 1 shows a slice data for a given patient.

2.2 Data Partitioning

This subsection describes data partitioning during model training. It also demonstrates how MR images of new patients are utilized during CT image prediction.

2.2.1 Data partition: Model training

CT image intensity threshold was utilized to partition the data into two major tissue types. Using 150HU CT image intensity as a limit, 50HU (is selected to take the delimiting value provided by Washington and Leaver [20]) overlap was allowed during parameter estimation process. The overlap of the major tissue types was motivated in order to minimize the effect of fuzzy boundary of the tissue types. Accordingly, CT image intensities in (-1024HU, 200HU) and (100HU, 3071HU] were assumed to represent non-bone and bone tissues. The minimum number of voxels for non-bone and bone tissues are 6214160 and 1292068.

2.2.2 MR images partition: CT image estimation

We are interested to predict CT images from MR images of new patients. Since we only have MR images of the new patients, there is a need to partition the MR images of the new patients into non-bone and bone tissues. There is poor bone tissue information on MR images and following that we estimate CT images for the new patients using a model trained on the full data, which is the data not partitioned into non-bone and bone tissues. The CT image intensity threshold and the estimated CT images are used to obtain MR data corresponding to the two major tissue types (non-bone and bone tissues). The trained models on the major tissue types are utilized to obtain the desired CT images of the new patients.

2.3 Statistical model: mixture of multivariate skew-normal model

Let $Y_{i1}$ and $\mathbf{Y}_{i2}=\left(Y_{i2},Y_{i3}\cdots,Y_{id}\right)^{\prime}$ represent voxel $i$ of CT image and its corresponding MR images. In our real data, d=5. A $d$-dimensional random vector $\mathbf{Y}_{i}=\left(Y_{i1},\mathbf{Y}_{i2}^{\prime}\right)^{\prime}$ is assumed to follow a multivariate skew-normal distribution $\mathcal{SN}\left(\mathbf{y}_{i}|\boldsymbol{\eta},\boldsymbol{\Sigma},\boldsymbol{\lambda}\right)$ with a $d$-dimensional location parameter vector $\boldsymbol{\eta}$, a $d\times d$-dimensional positive definite dispersion matrix $\boldsymbol{\Sigma}$, and a $d$-dimensional skewness parameter vector $\boldsymbol{\lambda}$. Its density can be given by

[TABLE]

where $\boldsymbol{\Sigma}^{-1/2}\boldsymbol{\Sigma}^{-1/2}=\boldsymbol{\Sigma}^{-1}$, $\Phi\left(\cdot\right)$ is a univariate standard normal distribution function, $i=1,2,3,\cdots,n$, and $n$ is the number of voxels. According to Lachos et al. [15], the stochastic representation of $\mathbf{Y}_{i}$ may be given by

[TABLE]

where

[TABLE]

In equation (2), $\mathbf{V}_{i}$ and $U_{i}$ are assumed to be independent. The notations $\mathbf{0}$, $\mathbf{1}$, and $\mathcal{HN}\left(u_{i}|0,1,\left(0,\infty\right)\right)$ in equation (3) represent $d$-dimensional zero vector, $d\times d$-dimensional identity matrix, and half-normal distribution, respectively. Using equation (2), a hierarchical model can be given by

[TABLE]

where

[TABLE]

Let $Z_{i}$ be a categorical random variable representing the underlying tissue types at voxel $i$. Define an indicator variable:

[TABLE]

where $k=1,2,\cdots,K$. The definition implies that $P\left(Z_{ik}=1\right)=P\left(Z_{i}=k\right)$. Let $P\left(Z_{i}=k\right)$= $\pi_{k}$, which represents the weight that the $i^{th}$ observation belongs to a tissue class $k$. To incorporate tissue heterogeneity into the statistical modeling, $\mathbf{Y}_{i}|Z_{ik}=1$ is assumed to follow a multivariate skew-normal distribution $\mathcal{SN}\left(\mathbf{y}_{i}|\boldsymbol{\eta}_{k},\boldsymbol{\Sigma}_{k},\boldsymbol{\lambda}_{k}\right)$. This means that $\mathbf{Y}_{i}$ follows a mixture of multivariate skew-normal distributions. Its density may be given by

[TABLE]

where

[TABLE]

The unknown parameters in $\boldsymbol{\Psi}$ are estimated from the independent observations $\mathbf{y}_{i}$.

2.4 Parameter estimation method

The log-likelihood function of the data $\mathbf{y}=\left(\mathbf{y}_{1},\mathbf{y}_{2},\cdots,\mathbf{y}_{n}\right)^{\prime}$ can be given by

[TABLE]

In general, there is no explicit analytical solution for $\arg\max\limits_{\boldsymbol{\Psi}}\log f\left(\mathbf{y}|\boldsymbol{\Psi}\right)$. However, iterative maximizing procedure under the idea of incomplete data via EM-algorithm can be used to obtain an optimal estimate of the parameters. Let $\mathbf{Z}_{i}$ be a $K$-dimensional column vector of $Z_{ik}$. Its realization is a $K$-dimensional vector consisting 1 at only one location and 0 at the remaining locations. The latent random vector $\mathbf{Z}_{i}$ follows a multinomial distribution with one trial and $P\left(Z_{ik}=1\right)=\pi_{k}$. Using the indicator variable $Z_{ik}$, the hierarchical model (4) can be extended to:

[TABLE]

where

[TABLE]

The observed data $\mathbf{y}$ is assumed to be incomplete data. It is augmented with the latent matrix $\mathbf{z}=\left(\mathbf{z}_{1},\mathbf{z}_{2},\cdots,\mathbf{z}_{n}\right)^{\prime}$ and the latent vector $\mathbf{u}=\left(u_{1},u_{2},\cdots,u_{n}\right)^{\prime}$ to form a complete dataset $\left(\mathbf{y},\mathbf{u},\mathbf{z}\right)$ in EM-algorithm framework. Assuming that $\left(\mathbf{Y}_{i},U_{i},\mathbf{Z}_{i}\right)$ is independent of $\left(\mathbf{Y}_{j},U_{j},\mathbf{Z}_{j}\right)$ for every $i\neq j$, the complete-data log-likelihood is given by

[TABLE]

where

[TABLE]

and $const$ is a constant function of parameters.

The E-step of EM-algorithm involves computing the expected value of the complete-data log-likelihood given $\mathbf{Y}$ and the current estimate $\boldsymbol{\Theta}^{old}$ of $\boldsymbol{\Theta}$. It may be given by the Q-function:

[TABLE]

Using equation (5), the expected value in equation (6) involves computing:

[TABLE]

The expected value $E\left[Z_{ik}|\mathbf{Y},\boldsymbol{\Theta}^{\text{old}}\right]$ can be given by

[TABLE]

where $\gamma_{ik}$ is the responsibility that the component $k$ of the mixture takes for explaining the observation $\mathbf{y}_{i}$. Let $\vartheta_{ik}=E\left[Z_{ik}U_{i}|\mathbf{Y},\boldsymbol{\Theta}^{\text{old}}\right]$, and $\psi_{ik}=E\left[Z_{ik}U^{2}_{i}|\mathbf{Y},\boldsymbol{\Theta}^{\text{old}}\right]$. Then $\vartheta_{ik}$ can be simplified as follows:

[TABLE]

Using similar procedure, $\psi_{ik}$ may be given by

[TABLE]

If equations (1) and (4) are satisfied, then using inverse matrix adjustment formula in [19] and matrix determinant lemma in [8],

[TABLE]

where $\mathcal{TN}\left(u|\mu,\sigma^{2},\left(0,\infty\right)\right)$ is a truncated normal with location parameter $\mu$, scale parameter $\sigma$ and support $\left(0,\infty\right)$. Based on the truncated normal probability distribution,

[TABLE]

where

[TABLE]

and $\phi\left({\cdot}\right)$ is a univariate standard normal density. The expected values:

[TABLE]

in equations (7) and (8) are obtained from equations (9) and (10) by replacing $\boldsymbol{\eta}$, $\boldsymbol{\xi}$, and $\boldsymbol{\Omega}$ with their corresponding estimates $\boldsymbol{\eta}_{k}^{old}$, $\boldsymbol{\xi}_{k}^{old}$, and $\boldsymbol{\Omega}_{k}^{old}$. The M-step of the EM-algorithm is given by

[TABLE]

and it is available in closed form. Using EM-algorithm, the estimates of the parameter $\boldsymbol{\Theta}$ can be updated as shown in Algorithm 1.

At $m$th iteration of the E-step, we need to compute the following expressions:

[TABLE]

In general, EM-algorithm converges to a local optimum. As a result, different initial values for the parameters are utilized during the estimation process to select the optimal estimates. A clustering method $K$-means has been employed to initialize the location parameters, the mixing coefficients, and the dispersion matrices. This clustering method is used to partition the observations into clusters in which each observation belongs to the cluster with the nearest mean. In this study, the clusters may represent tissue types or mixture of tissue types such as white matter, blood, water, fat, gray matter, air, cortical bone, cancellous bone, etc. We randomly initialise the remaining parameters of SGMM. Log-likelihood value can not be computed analytically for MRF [14] and therefore, we can not use it for selecting the optimal estimates. Following that we used mean squared error as criterion for selecting optimal estimates in SGMM in order to utilize the same criterion for the models used in this work. Two steps are employed during parameter estimation process. For a given number of tissue types, we estimated the parameters of the models and repeated this step to select the optimal parameter estimates using mean squared error. We selected the number of classes or tissue types through cross-validation using mean squared error. The stopping criterion for the convergence of the parameter estimation process is:

[TABLE]

with an upper limit $5\times 10^{-5}$, where $m$ denotes the iteration number of EM-algorithm.

2.5 Estimation of CT images

Let a $d$-dimensional vectors $\boldsymbol{\eta}$, $\boldsymbol{\nu}=\boldsymbol{\Sigma}^{-1/2}\boldsymbol{\lambda}$, and a $d\times d$ dispersion matrix $\boldsymbol{\Sigma}$ be partitioned as follows:

[TABLE]

The dimension of $Y_{i1}$, $\eta_{1}$ , $\nu_{1}$, and $\Sigma_{11}$ is a $1\times 1$. The random variable $Y_{i1}$ represents the $i^{th}$ voxel in CT image, and the random vector $\mathbf{Y}_{i2}$ denotes the corresponding voxel in MR images. If we assume that $\mathbf{Y}_{i}$ follows SGMM:

[TABLE]

then

[TABLE]

where

[TABLE]

According to Khounsiavash et al. [13], if equation (11) holds, then the probability density function of $Y_{i1}|\mathbf{Y}_{i2}=\mathbf{y}_{i2}$ can be given by

[TABLE]

where

[TABLE]

Proposition 2.1**.**

Using equation (12), the expected value of $Y_{i1}|\mathbf{Y}_{i2}=\mathbf{y}_{i2}$ can be given by

[TABLE]

Proof.

Let $\kappa\left(\mathbf{y}_{i2}\right)=\Phi\left(\boldsymbol{\tau}^{\prime}\boldsymbol{\Sigma}_{22}^{-1/2}\left(\mathbf{y}_{i2}-\boldsymbol{\eta}_{2}\right)\right)$ and $B=\boldsymbol{\nu}_{2}^{\prime}\left(\mathbf{y}_{i2}-\boldsymbol{\eta}_{2}\right)-\nu_{1}\eta_{1}$. The expected value of $Y_{i1}|\mathbf{Y}_{i2}=\mathbf{y}_{i2}$ can be given by

[TABLE]

where

[TABLE]

The result in equation (13) follows from well known properties of the truncated normal distribution and inverse matrix adjustment formula. ∎

Using equation (13), we can obtain the point estimator of $Y_{i1}$ by

[TABLE]

In this framework, the latent variable $Z_{i}$ represents the underlying tissue classes. The weight $P\left(Z_{i}=k|\mathbf{Y}_{i2}\right)$ can be computed using Bayes’ theorem. The expected value $E\left[Y_{i1}|\mathbf{Y}_{i2},Z_{i}=k\right]$ is obtained from equation (13) by indexing the parameters with $k$.

2.6 Model validation

The main focus of this work is to investigate CT image predictive quality of SGMM and GMM by partitioning the data into two major tissue types. It is also aimed at comparing the estimation quality of the partitioning approach with GMM, HMM, and MRF on the bone tissues.

We used leave-one-out cross-validation method to compare the predictive quality of the models. That is, we keep one head to validate the models and use the remaining heads for training the models. Using the threshold CT image intensity, CT image in a validation dataset is partitioned into non-bone and bone tissues. Let ${Y}^{\left(t\right)}_{i}$ and $\hat{Y}^{\left(t\right)}_{i}$ be CT image and its corresponding estimated CT image intensities in a given tissue region $t$. One can use the square and the absolute loss functions to assess the estimation cost. They can be given by

[TABLE]

where $t=1,2.$ Since the mean square error heavily weights the outliers, the mean absolute error (MAE) can be employed as a main tool to evaluate the estimation performance of the models. The MAE can be given by

[TABLE]

where $n_{t}$ is the number of voxels in partition $t$.

The peak signal-to-noise ratio (PSNR) can also be used to quantify the overall quality of the estimation. It takes square loss function into account through the mean squared error. The PSNR may be given by

[TABLE]

where $Y_{i}$ and $\hat{Y}_{i}$ represent CT image and the estimated CT image intensities at voxel $i$, respectively, and $M$ is the maximal intensity in CT image. We exploited patient-wise leave-one-out cross-validation and mean absolute error to compare the estimation quality of the models. One can also use peak signal-to-noise ratio to compare the estimation performance of the models. The better model has lower MAE and higher PSNR. Since MAE and PSNR are crude estimation quality measures, we utilised smoothed residual and absolute residual plots to further evaluate the estimation quality of the models through the tissues of the heads. A moving average over non-overlapping windows in CT image intensities can be used as a main tool to investigate and identify the model that outperforms in bone tissue estimation. Over the non-overlapping windows on ${Y}_{i}$ with a window size of 20 HU, the averages for ${Y}_{i}$, $\hat{Y}_{i}-{Y}_{i}$, and $|\hat{Y}_{i}-{Y}_{i}|$ over the windows can be computed to obtain the smoothed residual and absolute residual plots.

In addition to the above model performance evaluation methods, we assessed the performance of the models using Bland-Altman plot [32, 33], which is a graphical method to compare two measurements by plotting the differences between the two measurements $\hat{Y}_{i}-{Y}_{i}$ against their averages $(\hat{Y}_{i}+{Y}_{i})/2$ . If one of the two measurements is a reference measurement, then the differences can be plotted against the reference measurement [34] and that coincides with the smoothed residual plot via cross-validation. The main advantage of the Bland-Altman plot is that it reveals a relationship between the differences and the magnitude of the measurements, to examine possible systematic bias, and outliers.

We can also complement the MAE based performance evaluation of the methods by Wilcoxon signed-rank test, which is a non-parametric alternative to the paired Student’s t-test. To compare SGMM to every one of the remaining methods, the paired method-method differences in MAEs were examined by using the one-sided Wilcoxon signed rank test, with the null hypothesis that SGMM is the same as one of the other methods with respect to the MAE values and the research hypothesis that SGMM is better than the method being compared.

3 Results

A CT image intensity 100HU was utilized as a delimiting value during CT image estimation. The optimal parameter estimates were received for $K=8$ in GMM and $K=5$ for both HMM and MRF. In the case of SGMM and GMM*, we have received the optimal parameter estimates for $K=6$ for both major tissue types. Table 3 demonstrates a summary of mean absolute errors for the bone tissues and p-values of the Wilcoxon signed-rank test.

The rows of the table represent validation datasets. The table presents mean absolute errors received for the models. In terms of MAE, it is apparent that for each head the partitioning approach (SGMM and GMM*) and MRF had better performance than HMM and GMM. Moreover, the differences of the average MAEs show that the partitioning approach and MRF achieved better results than HMM and GMM. For instance, SGMM outperforms HMM with the method-to-method difference of average MAEs -17.27 and the standard deviation of the method-to-method differences of MAEs 8.23. The p-values of Wilcoxon signed-rank test show that SGMM outperforms HMM and GMM significantly on bone tissues.

The results in Table 3 show that the partitioning approach has noticeable outperformance on dense bone tissues (approximately with CT image intensities greater than 900HU according to Washington and Leaver [20]) as compared to the remaining models. Even though MRF is computationally expensive, it had better performance than HMM and GMM on dense bone tissues. The p-values of Wilcoxon signed-rank test reveal that SGMM has significantly better performance than the remaining methods except for GMM*∗* on dense bone tissues.

Table 3 demonstrates the estimation quality of the models on non-bone tissues. The best result was received for HMM. However, there is already a good contrast between soft tissues and air on MR images. The remaining models had similar behavior on non-bone tissues.

Table 3 presents the overall summary of CT image estimation quality. In comparison to the other models, we received better result for HMM. The reason is that HMM outperformed the other models on non-bone tissues. However, this is not the main interest in this work.

Table 3 demonstrates the prediction accuracy of the models in terms of PSNR. The results show that the models had similar behavior. On the bone tissues, PSNR has shown that the models have similar estimation performance.

On the basis of average, mean absolute error was utilized to compare CT image prediction accuracy of the models. Smoothed absolute residual plots were also employed to assess the estimation quality of the models. In comparison to MAE, smoothed absolute residual plots are powerful to evaluate the estimation performance of the models through the tissues of the heads. Figure 2 presents smoothed absolute residual plots for the models. The absolute residuals were averaged over non-overlapping windows in CT image intensities with window size 20HU.

It is clear from Figure 2 that none of the models outperformed throughout the tissues of the head. However, SGMM and GMM*∗* outperformed the other models on dense bone tissues. Figure 3 shows smoothed residual plot. The deviations of observed CT image intensities from the estimated CT image intensities were exploited to obtain average residuals over non-overlapping windows in CT image intensities with window size 20HU. It is evident from the plot that bone tissues were underestimated. However, the partitioning approach has improved underestimation of bone tissues.

The Bland-Altman plots of the methods are shown in Figure 4. It shows the bias of the methods in estimating the CT images. The bias is higher for higher CT image values. However, the partitioning approach has improved the bias in bone tissue estimation as compared to the remaining methods. The Bland-Altman plots also show higher variability on lower CT image values. In comparison to 3, we observe that the higher variability on lower CT image values in Figure 4 is due the estimated CT image values. Notice, however, that this work is mainly concerned with improving bone tissue estimation.

The partitioning approach had better performance in tracing bone tissues. Bone tissues appear as bright white on CT images, see Figure 5. The figure shows slices of CT image and its corresponding predicted slices for a representative patient. The first row in the figure where the images are marked by red colored box clearly shows that the partitioning approaches are better in identifying bone tissues.

We presented the prediction errors in Figure 5, which corresponds to the predicted slices in Figure 6. It can be seen that the images of the prediction errors corresponding to the bone tissues appear darker for the partitioning approach: GMM*∗* and SGMM.

4 Discussion

Statistical model for voxel-wise CT image estimation from MR images have been presented and evaluated using cross-validation on five datasets. Kuljus et al. [14] and Johansson et al. [10] have used voxel-wise estimation approach to study CT image estimations. Existing works suggest that the estimation quality on the bone tissues is poor. This study was aimed at probing the estimation of CT images by partitioning the data into two major tissue types: non-bone and bone tissues. Specifically, the focus of the study was to obtain improved bone tissue estimations. We proposed SGMM to relax the distributional assumption utilised by Kuljus et al. [14] and Johansson et al. [10]. The model was motivated to take the asymmetrical distribution that could arise from the partitioning of the data into account. We also used GMM in addition to SGMM to examine the effect of the distributional nature of the partitioned data on the estimation process. The study was also aimed at comparing CT image estimation performance of SGMM, HMM, GMM, and MRF on the bone tissues. In MRF and HMM, spatial dependence between neighboring voxels has been taken into account.

EM-algorithm was developed for SGMM parameter estimation. EM-gradient algorithm was used to estimate the parameters of MRF while EM-algorithm was utilized to estimate the parameters of GMM and HMM. For MRF, the updates of the parameters at M-step of EM-algorithm were difficult to obtain in an explicit form and thereby, a gradient based optimization was utilized during parameter estimation process. Moreover, Gibbs sampling was used at E-step of the algorithm. Thus, the estimation process was expensive in MRF. Unlike the other models, log-likelihood function in MRF involves Gibbs field and it is not computable. This means that log-likelihood based selection of optimal parameter estimate is not feasible analytically. Consequently, mean squared error was employed in selecting the optimal parameter estimates of the models.

Table 3 demonstrated that HMM outperformed the other models. This was essentially due to its best estimation performance on non-bone tissues, see Table  3. According to Karlsson et al. [12], the key task in CT image estimation from MR images is to obtain an improved bone tissue estimation. Table 3 revealed that GMM* (GMM trained on each major tissue type), SGMM, and MRF had better prediction accuracy than HMM and GMM on the bone tissues. In addition, Table 3 shows that GMM* and SGMM had noticeable outperformance on dense bone tissues. Based on mean absolute error, HMM and GMM perform similarly on the bone tissues.

To provide statistical evidence for the CT image estimation performance of SGMM in comparison with the remaining methods, the Wilcoxon signed-rank tests have been carried out. The tests has shown that SGMM has better performance on bone tissues than HMM and GMM. In addition, the Wilcoxon signed-rank tests provided an evidence that SGMM has significant outperformance than MRF, HMM, and GMM on dense bone tissues.

The skewness assumption allowed to recognize skewness in the partitions of the data. The estimates of the skewness parameters demonstrated that the partitions have skewness property and the nature of skewness was dependent on the subtissue types. The estimates of the skewness parameters ranged from -1.54 to 3.03. In this particular application, however, the skewness assumption did not improve CT image estimation quality as compared to the symmetric assumption in GMM*, see Table  3–3.

Figure 2 shows that the models performed better on soft tissues, that is tissues having CT image intensity closer to 0HU. On the other hand, the models had weaker prediction accuracy on the two extremes that is on air and bone tissues. This pattern of residual plots have been observed in [10, 14]. Moreover, the figure shows that none of the models outperformed throughout the tissues of the head. Table 3–3 demonstrate that the predictive accuracy of the models was dependent on the heads. That is the results were not uniform over the heads. This might have a problem in real applications and needs a further investigation.

The bias of the methods in estimation of the CT images was manifested in the smoothed residual and Bland-Altman plots in Figures 3 and 4. Though the bias is higher on bone tissues, it was improved by our approach as compared to the remaining methods. This was the main concern of this work. The Bland-Altman plots also demonstrated higher variability of the estimation on lower CT image values.

5 Conclusions

In this study, we examined CT image estimation by partitioning the data into two major tissue types. This partitioning approach is an efficient way to get a good quality CT image substitute with improved estimation of bone tissues. Moreover, the SGMM and the developed algorithm to estimate its parameters is general and can be applied to other applications.

Acknowledgments

This work is supported by the Swedish Research Council grant (Reg. No. 340-2013-5342). The authors thank Adam Johansson and Thomas Asklund for providing us data and David Bolin for providing us Mathlab code for MRF. Moreover, the authors thank Kristi Kuljus for HMM results. The computations were performed on resources provided by the Swedish National Infrastructure for Computing (SNIC) at High Performance Computing Center North (HPC2N).