Accurate and Fast reconstruction of Porous Media from Extremely Limited Information Using Conditional Generative Adversarial Network

TL;DR

This paper introduces a deep learning framework using conditional GANs to accurately and quickly reconstruct full 3D porous media images from limited sub-region data, addressing challenges of incomplete information.

Contribution

The study develops a novel CGAN-based method with tailored objective functions for reconstructing entire images from small partial inputs, improving accuracy and speed.

Findings

Reconstruction accuracy demonstrated on various porous materials.

Method achieves fast processing time (~0.08s for 128x128 images).

Reconstruction diversity enabled by Gaussian noise.

Abstract

Porous media are ubiquitous in both nature and engineering applications, thus their modelling and understanding is of vital importance. In contrast to direct acquisition of three-dimensional (3D) images of such medium, obtaining its sub-region (s) like two-dimensional (2D) images or several small areas could be much feasible. Therefore, reconstructing whole images from the limited information is a primary technique in such cases. Specially, in practice the given data cannot generally be determined by users and may be incomplete or partially informed, thus making existing reconstruction methods inaccurate or even ineffective. To overcome this shortcoming, in this study we proposed a deep learning-based framework for reconstructing full image from its much smaller sub-area(s). Particularly, conditional generative adversarial network (CGAN) is utilized to learn the mapping between input (partial image) and output (full image). To preserve the reconstruction accuracy, two simple but effective objective functions are proposed and then coupled with the other two functions to jointly constrain the training procedure. Due to the inherent essence of this ill-posed problem, a Gaussian noise is introduced for producing reconstruction diversity, thus allowing for providing multiple candidate outputs. Extensively tested on a variety of porous materials and demonstrated by both visual inspection and quantitative comparison, the method is shown to be accurate, stable yet fast ($\sim0.08s$ for a $128 \times 128$ image reconstruction). We highlight that the proposed approach can be readily extended, such as incorporating any user-define conditional data and an arbitrary number of object functions into reconstruction, and being coupled with other reconstruction methods.

Equations

$$\underset{G}{\min }\underset{D}{\max }{L}_{GAN}\left(G,D\right)=E_{\mathbf{y}\sim p_{data}\left(\mathbf{y}\right)}\left[\log D\left(\mathbf{y}\right)\right]$$

$$+E_{\mathbf{z}\sim p_{\mathbf{z}}\left(\mathbf{z}\right)}\left[\log \left(1-D\left(G\left(\mathbf{z}\right)\right)\right)\right]$$

$$\underset{G}{\min }\underset{D}{\max }{L}_{CGAN}\left(D,G\right)=E_{\mathbf{x}\sim p_{data}\left(\mathbf{x}\right),\mathbf{y}\sim p_{data}\left(\mathbf{y}\right)}\left[\log D\left(\mathbf{x},\mathbf{y}\right)\right]$$

$$+E_{\mathbf{x}\sim p_{data}\left(\mathbf{x}\right),\mathbf{z}\sim p_{\mathbf{z}}\left(\mathbf{z}\right)}\left[\log \left(1-D\left(\mathbf{x},G\left(\mathbf{x},\mathbf{z}\right)\right)\right)\right]$$

$${L_{D}}=-\left\{\begin{array}[]{l}{E_{\mathbf{x}\sim{p_{data}}\left(\mathbf{x}\right),\mathbf{y}\sim{p_{data}}\left(\mathbf{y}\right)}}\left[{\log D\left({\mathbf{x},\mathbf{y}}\right)}\right]+\\ {E_{\mathbf{x}\sim{p_{data}}\left(\mathbf{x}\right),\mathbf{z}\sim{p_{\mathbf{z}}}\left(\mathbf{z}\right)}}\left[{\log\left({1-D\left({\mathbf{x},G\left({\mathbf{x},\mathbf{z}}\right)}\right)}\right)}\right]\end{array}\right\}$$

$$L_G=E_{\mathbf{x}\sim p_{data}\left(\mathbf{x}\right),\mathbf{z}\sim p_z\left(\mathbf{z}\right)}\left[\log \left(1-D\left(\mathbf{x},G\left(\mathbf{x},\mathbf{z}\right)\right)\right)\right]$$

$$L_{L_1}={\Vert \mathbf{x}-G\left(\mathbf{x},\mathbf{z}\right)\Vert }_1,$$

$$L_{pattern}={\Vert y_{pattern}-{\left(\mathbf{G}\left(\mathbf{x},\mathbf{z}\right)\right)}_{pattern}\Vert }_2^2$$

$$L_{porosity}={\Vert y_{porosity}-{\left(G\left(\mathbf{x},\mathbf{z}\right)\right)}_{porosity}\Vert }_2^2$$

$$\displaystyle\begin{array}[]{c}{L_{total{\rm{}}loss}}={L_{CGAN}}\left({G,D}\right)+{\lambda_{{L_{1}}}}{L_{{L_{1}}}}\\ +{\lambda_{pattern}}{L_{pattern}}+{\lambda_{porosity}}{L_{porosity}},\end{array}$$

Full text

Porous media; Microstructure reconstruction; Deep learning; Conditional generative adversarial network (CGAN); Extremely limited information

††preprint: APS/123-QED

I Introduction

Porous media, such as sandstone, soil, alloy, and composite abound in nature and synthetic situations and have play a critical role in a variety of engineering applications. Hence, their understanding and modelling is of significant importance Torquato (2013); Sahimi (2011).Despite the advance of three-dimensional (3D) imaging techniques like computed tomography (CT) Li et al. (2018a); Wang et al. (2018a); Bostanabad et al. (2018) and scanning electron microscope (SEM) Tahmasebi et al. (2015), however, in many cases, only limited data is available for analysis. They could be 3D incomplete data, several two-dimensional (2D) slices and even a few statistics Yeong and Torquato (1998a, b). Therefore, reconstructing 3D full images from these limited data has been a major technique in such situations.

During the past decades, various reconstruction methods have been developed Yeong and Torquato (1998a, b); Rozman and Utz (2001); Pant et al. (2014); Jiao et al. (2008, 2009); Chen et al. (2015); Gerke and Karsanina (2015); Gerke et al. (2014); Karsanina and Gerke (2018); TANG et al. (2009); Chen et al. (2014); Gao et al. (2016); Feng et al. (2018a); Ju et al. (2017, 2014, 2018); Okabe and Blunt (2005); Gao et al. (2015); Ding et al. (2018); Mariethoz et al. (2010); Tahmasebi and Sahimi (2012, 2013, 2016a, 2016b); Tahmasebi (2017); Bostanabad et al. (2016a, b); Feng et al. (2018b); Mosser et al. (2017, 2018a, 2018b); Laloy et al. (2017, 2018); Wang et al. (2018b); Li et al. (2018b, 2019), and popular algorithms include optimization-based method Yeong and Torquato (1998a, b); Rozman and Utz (2001); Pant et al. (2014); Jiao et al. (2008, 2009); Chen et al. (2015); Gerke and Karsanina (2015); Gerke et al. (2014); Karsanina and Gerke (2018); TANG et al. (2009); Chen et al. (2014); Gao et al. (2016); Feng et al. (2018a); Ju et al. (2017, 2014, 2018), multi-point statistics (MPS) Okabe and Blunt (2005); Gao et al. (2015); Ding et al. (2018), direct sampling (DS) Mariethoz et al. (2010), CCSIM Tahmasebi and Sahimi (2012, 2013, 2016a, 2016b); Tahmasebi (2017), machine learning and deep learning based method Bostanabad et al. (2016a, b); Feng et al. (2018b); Mosser et al. (2017, 2018a, 2018b); Laloy et al. (2017, 2018); Wang et al. (2018b), and superdimension method recently proposed Li et al. (2018b, 2019). It is well known that generally the prerequisite of this reconstruction methodology is that the 2D training image (TI) needs to meet the requirements of stationarity and ergodicity, in other words, 2D image is able to statistically represent the main characteristic of the entire 3D structure. Despite considerable research on TI selection Mirowski et al. (2009); Boisvert et al. (2007); Gao et al. (2017), however, in practice the 2D images or data are generally not determined by users, and they may be incomplete or partially applicable Shen et al. (2015); Mariethoz and Renard (2010); Sokat et al. (2018), and thus cannot be directly used as a representative of 3D images. For instance, loss of data or information in 2D images is a universal issue in Earth Sciences and is the primary cause of (hydro) geological uncertainty Mariethoz and Renard (2010). Clearly, directly reconstructing 3D image from its imperfect 2D images may be infeasible. Instead, to first recover the 2D full image using the limited information and then apply it to 3D reconstruction could be an alternative solution.

Even though 2D image reconstruction can serve as the preparation of the subsequent 3D reconstruction, in fact, reconstruction of 2D image and the analysis based on which can be a distinctive topic. It is noteworthy that practically the 2D images obtained by optical microscope or SEM still play a key role in numerous researches. Notice that in some circumstances acquisition of 2D images could also be tough and highly expensive Abdollahifard et al. (2016); Ju et al. (2018); Semnani and Borja (2017). For example, study of nano-scale pore in tight porous materials such as shale generally requires a huge number of high-resolution 2D images for comprehensive analysis, because usually a single imaging field of view (FOV) of such sample is too small to represent the entire material Semnani and Borja (2017). Hence, it is of critical importance and great interest to employ the acquired data (e.g., several small FOVs) for quick reconstruction and accurate investigation, which may save both time and imaging cost.

Considering the above two aspects, effectively utilizing the given (obtained) data and correspondingly developing accurate 2D/3D reconstruction methods still remain an outstanding problem. Figure 1 shows a schematic of 2D reconstruction of such process. In addition, another related issue is the diversity of reconstruction results. Since the inherent essence of this inverse problem generally enables that more than one solutions are acceptable, the reconstruction algorithm is expected to be not only accurate and fast, but also able to stably provide comparable candidate solutions, thus allowing for user selection.

At present, potential methods for this reconstruction conundrum could be the variants of DS Mariethoz et al. (2010) and CCSIM Tahmasebi and Sahimi (2012, 2013, 2016a, 2016b); Tahmasebi (2017), and we notice that, the performance of these MPS-like methods heavily rely on the proportion of informed data, i.e., the more data there is, the better performance they may achieve. In the case of extremely limited information here, inaccurate reconstruction and evident artifacts (unnatural structure) may easily arise (see results in Sec.IV).

Recently, the advent of machine learning/deep learning techniques has brought new inspirations and insights in a variety of domains Bostanabad et al. (2016a, b); LeCun et al. (2015); Esteva et al. (2017); Chen et al. (2017, 2018a); Ren et al. (2019); Goodfellow et al. (2014); Mirza and Osindero (2014); Karimpouli and Tahmasebi (2019); Karimpouli and Tahmesbi (2019); Tahmasebi et al. (2017); Chen et al. (2018b); Cang et al. (2017); Chan and Elsheikh (2018a, b); Feng et al. (2018b); Mosser et al. (2017, 2018a, 2018b); Laloy et al. (2017, 2018); Wang et al. (2018b); Zhu et al. (2018). Notably, there have been an increasing number of such techniques utilized in the reconstruction and analysis of materials Karimpouli and Tahmasebi (2019); Karimpouli and Tahmesbi (2019); Tahmasebi et al. (2017); Chen et al. (2018b); Cang et al. (2017); Chan and Elsheikh (2018a, b); Feng et al. (2018b); Mosser et al. (2017, 2018a, 2018b); Laloy et al. (2017, 2018); Wang et al. (2018b). Recent advances include decision tree method Bostanabad et al. (2016a, b), 2D generative adversarial network (GAN) and 2D conditional generative adversarial network(CGAN) Chan and Elsheikh (2018a, b); Feng et al. (2018b), 3D GAN and 3D CGAN Mosser et al. (2017, 2018a, 2018b); Laloy et al. (2017, 2018), and CNN-based method Wang et al. (2018b). The prevalence of deep learning-based method is due to that by training a neural network using numerous samples (pairs of input and output), it can find a general mapping from input to output. Once neural network is trained, its inference (reconstruction) can be very quick.

In this paper, we aim to propose a deep learning-based framework for reconstructing porous media from extremely limited information. Specially, CGAN is employed to learn a mapping between input (partial image) and output (full image). To preserve the reconstruction accuracy, two simple but effective objective functions (aka, loss functions) are proposed and then coupled with another two loss functions to jointly constrain the training procedure. Additionally, considering the intrinsic nature of this inverse problem that generally more than one reconstructions are reasonable, a Gaussian noise is introduced to produce reconstruction diversity, thus allowing for providing multiple candidate outputs. According to the extensive tests on a variety of porous media and the both visual and quantitative comparisons, our method is demonstrated to be accurate yet stable. Moreover, when given an input, the proposed method can render instant reconstruction on CPU ($\sim 0.08s$ for a $128\times 128$ image), which achieves $\sim 20$ speedup factor compared with conventional method DS. We remark that apart from the ability to solve the problem of incomplete data, our approach can be readily extended, such as incorporating an arbitrary number of object functions of any types into reconstruction, the ability to incorporate any user-define conditional data, and being coupled with other reconstruction methods.

The rest of this paper is organized as follows: Section II details the reconstruction framework of porous media, including fundamental of CGAN, introduction of noise, and the loss functions. Section III describes the assessment methods for reconstruction. Results and comparisons are demonstrated in Sec. IV. In Section V, we make concluding remarks.

II Reconstruction of porous media using conditional generative adversarial network

In this section, we will present the details on the reconstruction of porous media using conditional generative adversarial network (CGAN), including primary principle of CGAN, design of network architectures, and loss functions.

II.1 Principle of CGAN

Owing to CGAN Mirza and Osindero (2014) is the conditional version of GAN Goodfellow et al. (2014), we would like to first introduce GAN. In particular, GAN consists of two adversarial sub-networks, one is generator $G$ and the other is discriminator $D$. Especially, they battle in a two-player min-max game in which the generator $G$ tries to generate realistic data from given input to fool discriminator $D$, whereas $D$ attempts to identify the fakes by $G$ and real data (target). The goal of $G$ is to learn a mapping $G(\mathbf{z})$ that maps prior noise ${p_{z}}\left(\mathbf{z}\right)$ to the real data $\mathbf{y}$, while the output of discriminator $D(\mathbf{y})$ is to give the probability that $\mathbf{y}$ is from training data rather than the fake data generated by $G$. Both $G$ and $D$ are trained alternately to optimize the following objective function:

[TABLE]

where $G$ tries to minimize this expression and in the same time $D$ tries to maximize it, thus indicating the adversarial conception in GAN. Notably, at the very beginning, the abilities of both $G$ and $D$ are quite weak; with iterations, they gradually evolve to be powerful and finally reach Nash equilibrium Goodfellow et al. (2014), in which $G$ is able to produce realistic data which cannot be recognized by $D$. Once trained, $D$ is discarded and only $G$ is used to transform input to its expected output. In general, the use of initial GAN is to generate realistic data using noise distribution such as Gaussian or rand noise from the given samples, which could be of special use in the situations that data is lacking.

As an improved version of GAN, the CGAN allows for incorporating conditional data as an external input $\mathbf{x}$, rather than only a noise distribution $\mathbf{z}$, as demonstrated in Fig.2.

Therefore, the modified objective function is given as:

[TABLE]

Actually, in terms of different tasks, the noise in CGAN could be added or not. Especially, the noise could even be dropped in the case of one-to-one image translation problems Isola et al. (2017). However, in this paper, the diversity of reconstruction is mainly due to the introduction of noise. Hence, the noise is reserved and fused together with $G$ (see more details presented in subsection II.2.3).

II.2 Network architectures

In the following, we will elaborate the architectures of generator G and discriminator D, as well as the injection of noise into G.

II.2.1 Architecture of generator

Specially, in general image-to-image tasks, U-Net network architecture is frequently employed, for its merit that relatively fewer parameters and multi-scale characteristic for feature extraction. Owing to the nature of reconstruction of porous media is also an image-to-image task, we mainly follow the design of BicycleGAN Zhu et al. (2018), as it is a representative U-Net based framework. Figure 3 depicts the main architecture of G. Specifically, this network performs two main steps: encoding process by encoder and decoding process by decoder. These two procedures could be viewed as the nonlinear down-sampling and up-sampling of input image. In this work we focus on $128\times 128$ images, so starting from the size of $128\times 128$, the input image is convoluted and down-sampled gradually to $1\times 1$ code by the sampling factor 2, and then deconvoluted and up-sampled back to $128\times 128$ for output by the same factor 2. Each convolutional layer or deconvolutional is followed by nonlinear activation layer (ReLU or Leaky ReLU) and instance normalization (IN) layer. In general, the encoding and decoding processes of U-Net are symmetric, which means that the shape of feature maps in the symmetric position are the same. Thus, a skip connection is usually added to assist decoding process by introducing the feature map information of encoding process.

II.2.2 Architecture of discriminator

The design of discriminator $D$ is relatively simpler than generator G, as shown in Fig. 4. It is composed of five convolutional layers, each of them is followed by Leaky ReLU (LReLU) layer and instance normalization (IN) layer. As aforementioned, $D$ is trained to distinguish the real data $\mathbf{y}$ and fake data $G(\mathbf{z})$. Hence, the input of $D$ is the pair of conditional data $\mathbf{x}$ and real data $\mathbf{y}$, or that of conditional data $\mathbf{x}$ and fake data $G(\mathbf{z})$ during training process. By using the sigmoid function and an additional average function, the output of $D$ is transformed to a probability between [math] to $1$, which indicates how real the data is. The D is trained to recognize real data $\mathbf{y}$ as probability $1$ and fake data $G(\mathbf{z})$ as probability [math]. When reaching the final balance with $G$, both real and fake data are identified as probability $0.5$.

II.2.3 Noise injection

It is worth noting that the goal of injection of random noise into generator $G$ is to introduce its output diversity, for providing multiple choices for users due to the inherent essence of this ill-posed problem. In general, there are two alternatives of noise injection: i) adding noise in the first layer (Fig. 5 a) and ii) in each layer of the encoder (Fig. 5 b). As demonstrated in the literature Zhu et al. (2018), the noise injected all layers in the encoder leads to a slightly better performance. Hence, in the proposed method we utilize the design in Fig. 5 b.

In our work, we use a Gaussian noise with a shape $n_{z}\times 1\times 1$, which is first spatially replicated to same height and width of the current layer of encoder, and then is concatenated with this layer along channel dimension. For instance, assume the shape of current layer is $C\times H\times W$, the Gaussian noise $n_{z}\times 1\times 1$ is spatially replicated to $n_{z}\times H\times W$ along H and W dimension. After concatenation along channel dimension, the shape of resulting concatenated layer will be $(C+n_{z})\times H\times W$.

II.3 Loss function

Similar to other optimization-based approaches like simulated annealing Yeong and Torquato (1998a, b); Rozman and Utz (2001); Pant et al. (2014); Jiao et al. (2008, 2009); Chen et al. (2015); Gerke and Karsanina (2015); Gerke et al. (2014); Karsanina and Gerke (2018); TANG et al. (2009); Chen et al. (2014); Gao et al. (2016); Feng et al. (2018a); Ju et al. (2017, 2014, 2018), the goal of deep learning-based method is to minimize the loss function, which is used to evaluate the discrepancy between network prediction and target. On the basis of Eq. 2, the loss of discriminator D can be equivalently concluded to minimize the equation as follows:

[TABLE]

In what follows, we would like to detail the loss function of generator G, since it significantly determines the quality of generated results. Specially, the total loss function of $G$ is composed of four individual loss functions, namely, CGAN loss $L_{G}$, L1 loss $L_{L1}$, and two proposed loss functions for this reconstruction task, i.e., pattern loss $L_{pattern}$ and porosity loss $L_{posority}$.

CGAN loss $L_{G}$ comes from the fundamental of CGAN framework, representing how close output of $G$ is to the target when discriminated by $D$ Goodfellow et al. (2014). In terms of Equation Eq. 2, the loss $L_{G}$ is defined as:

[TABLE]

L1 loss $L_{L1}$, defined as a sum of pixel-wise absolute value difference between $G^{\prime}s$ output $G(\mathbf{x},\mathbf{z})$ and input conditional data $\mathbf{x}$, is given by:

[TABLE]

where $\mathbf{z}$ is Gaussian noise. This loss function is utilized to ensure that the output $G(\mathbf{x},\mathbf{z})$ keeps the same conditioning data contained in $\mathbf{x}$, including both value and position.

The third one is pattern loss $L_{pattern}$, proposed to quantify the mean squared error (MSE) of pattern distributions in the respective images of $G^{\prime}s$ output $G(\mathbf{x},\mathbf{z})$ and target $\mathbf{y}$. This loss function is leveraged to describe the local texture difference in two images, and the definition is:

[TABLE]

Particularly, the pattern in an image is defined as the multi-point configuration captured by a template. The calculation of pattern loss is straightforward: i) by using a $N\times N$ template, scan the image and collect all the patterns; ii) flatten each pattern, and obtain its corresponding binary code and then convert it to decimal number; iii) the occurrence of each decimal number is counted and then is normalized (divided by the total number of patterns in the image), and thus the probability distribution of patterns is obtained; iv) calculate the Euclidean distance between the two distributions of target and reconstruction. Figure 6 gives the main steps of such process.

The last loss function is porosity loss $L_{posority}$, which is presented to identify the disagreement of porosity of output $G(\mathbf{x},\mathbf{z})$ and target $\mathbf{y}$, and thus maintaining the porosity agreement during reconstruction. It is given as follows:

[TABLE]

The total loss is a weighted sum of above four loss functions and is defined as:

[TABLE]

where the hyper-parameters $\lambda_{{L_{1}}}$, $\lambda_{pattern}$ and $\lambda_{porosity}$ control the relative importance of each term.

III Assessment methods for reconstruction

In this paper, to verify the performance of the proposed method, porosity and three morphological functions, namely, two-point correlation function Torquato (2013), lineal-path function Lu and Torquato (1992) and two-point cluster function Torquato et al. (1988), are employed. Figure 7 illustrates the definition of these functions.

Two-point correlation function $S_{2}(\mathbf{r})$ gives the probability that two points separated by a distance $r=\|\mathbf{r}\|$ both lie in the same phase, indicating spatial correlation of two points. For statistically isotropic and homogeneous media, it only relies on the distance between the point pair. Therefore, for briefness $r$ is dropped.

Similar to the $S_{2}$, the two-point cluster function $C_{2}$ presents the probability that two points both lie in the same cluster. In terms of its definition, it embodies higher-order morphological information than $S_{2}$.

The other descriptor is lineal-path function $L$, which is the probability that a segment entirely lies in the same phase. It encodes connectedness information of medium along a lineal path.

IV Results and discussion

Here, we focus on 2D two-phase structures, while with slight modifications, extensions of multi-phase reconstruction and 3D reconstruction are also straightforward. To ascertain the performance of our method CGAN, we test it on four types of porous media, covering high or low porosity, and isotropy or anisotropy.

IV.1 Dataset

Notice that in this work, in terms of each category of porous medium, an associated dataset is made, which encompasses 600-1000 images. Especially, 70$\%$ samples in each dataset is randomly chosen as training set and the rest 30$\%$ is as testing set. Figure 8 presents six examples in the dataset of silica material, each sample is a pair of input (partial image) and target (full image).

In all of our reconstruction tasks here, hyper-parameter $\lambda_{{L_{1}}}$, $\lambda_{pattern}$ and $\lambda_{porosity}$ in Eq. 8 are empirically set to $10$, $5.0\times 10^{5}$ and $1.0\times 10^{3}$, respectively. For the tradeoff between accuracy and efficiency of pattern loss, template size $N$ in calculation of pattern loss is set to $3$. In addition, batch size and channel $n_{z}$ of noise are respectively set to $2$ and $8$. For both $G$ and $D$, we use Adam optimizer Kingma and Ba (2014) with an initial learning rate $0.0002$, and a linear decay for stable training.

IV.2 Results and comparisons

In this subsection, we present reconstruction results and comparisons of four porous media, including isotropic materials such as silica, battery material, sandstone, and an anisotropic medium. In particular, porosity and three morphological functions, i.e., $S_{2}$, $L$, and $C_{2}$ are employed to evaluate the reconstruction accuracy. For each medium, $20$ realizations are generated and an additional average of statistics (morphological functions and porosity) over them is also presented.

IV.2.1 Silica reconstruction

First, we reconstructed an isotropic porous material, i.e., silica in a rubber matrix with a size of $128\times 128$ and a low porosity ${\phi_{target}}=0.0962$. Figure 9 demonstrates the input, three randomly selected reconstructions, as well as the target. Notice, the informed data in the input is a $26\times 26$ square in the top-left area of the target, only accounting for $\sim 4\%$ data in this image. However, the reconstructions are still visually indistinguishable from the target. It can be obviously seen that, the hard data in the input is well honored in all of the reconstructions (marked in orange rectangles), while the rest of the image keeps good diversity, which allows for multiple selections for users.

Moreover, in addition to visual inspection, we further quantitatively compare the reconstruction accuracy of the proposed method. Figure 10 depicts the quantitative comparison of statistical functions and porosities between $20$ reconstructions, their average, and the target. Good agreement can be observed between reconstructions and target, and the average over $20$ realizations excellently matches the target, demonstrating the accuracy of proposed method. Additionally, the small biases of functions between reconstructions and target also indicates the stability of our method. The porosity distribution (Fig. 10 a) on $20$ reconstructions is ${\phi_{CGAN}}=0.099\pm 0.003$ $(Mean\pm Standard\;Deviation)$ while the target is ${\phi_{target}}=0.0962$, their accordance also manifests the accuracy and robustness of the proposed method.

Notably, once trained, our method only takes $\sim 0.08s$ for reconstruction when running on an Intel i7-4790K ( $4.00$ GHz) CPU. It is also worth noting that this input image (Fig. 9 a) may be regarded as a high-resolution image with small FOV and by using our method, its much larger FOV with high resolution can be accurately recovered. Meanwhile, according to the input and reconstructions (Fig. 9), some of white clusters (marked in green rectangles) in reconstruction are much bigger than the given ones, therefore indicating the additional information is indeed introduced by our method.

IV.2.2 Battery material reconstruction

In addition to one subarea reconstruction, our method is also verified on an isotropic battery material Ananyev et al. (2018) with four tiny $13\times 13$ subareas informed and target porosity ${\phi_{target}}=0.294$, as shown in Fig. 11 a and Fig.11 e. The purpose of this experiment is to mimic the extreme circumstances in aerospace Shen et al. (2015), geoscience Mariethoz and Renard (2010); Sokat et al. (2018), etc., that only several incomplete parts of data may be available. Again, according to the comparisons in Fig. 11, the reconstructions are visually comparable, and in terms of both porosity (${\phi_{CGAN}}=0.306\pm 0.022$ ) and statistical functions, the agreement is excellent.

We would like to emphasize that the amount of informed data in the input image of this material is the same as that of silica, however, here the standard deviation $0.022$ of its reconstruction porosities is much higher, in comparison with that of the silica reconstruction ($0.003$). This phenomenon can be also be evidently observed from the larger vibrations of the three descriptors in Fig. 11(f)-(h). Actually, this is expected since this material is more complex in both connection and geometry than those of silica material.

IV.2.3 Sandstone reconstruction

It is useful to compare the performances of our method CGAN and another existing method, i.e., a variation of $DS$ (here also called $DS$ for convenience). As presented in Fig. 12 a, we use a sandstone image with a relatively larger region ($128\times 20$) of known data for the demonstration purpose. Obviously, In light of both visual and quantitative comparisons in Fig. 12, the performance of CGAN significantly surpasses that of DS. As can be observed in Fig. 12 d, the reconstruction by DS is unnatural, in which the pore size is severely underestimated and the connectedness is also poor, compared with the target image. Specially, the reasons for this are twofold: i) the reconstruction mechanism of sequential simulation of DS can readily cause error accumulation and thus make reconstruction inaccurate; ii) more importantly, the nature of this method is generally directly repeating the given data (usually in the manner of patterns) into the rest of unknown part, rather than generating or introducing additional realistic information. As can be seen Fig. (12 d), rectangles marked in red and blue demonstrate the repeat of two patterns of hard data in input.

By contrast, the two reconstructions of CGAN (Fig. 12 b and Fig. 12 c) are both visually and quantitatively in accordance with target, unreal structures could be hardly recognized. Also, these reconstructions present seamless transition between the edge of hard data and unknown data. Besides, reconstruction of CGAN only takes $\sim 0.08s$, while DS takes $\sim 1.6s$ on the same CPU, achieving a $\sim 20$ speedup factor.

IV.2.4 Anisotropic reconstruction

We further apply our method to reconstruction of an anisotropic porous material Bostanabad et al. (2016a), as demonstrated in Fig. 13 e. The particularity of this medium is the pore along southeast is significantly longer than other directions. Clearly, even though $128\times 10$ hard data is given (Fig. 13 a), the reconstructions are still realistic at visual level, and the hard data is also well preserved. The reconstruction porosity distribution is ${\phi_{CGAN}}=0.379\pm 0.064$, which is close to the target ${\phi_{target}}=0.393$.

As can be seen in Fig. 13 f-h, the averaged $S_{2}$ and $L$ of reconstructions are in good agreement with those of target, whereas higher-order function $C_{2}$ exhibits slight bias, which especially enlarges with the increase of the distance between two pixels. Remarkably, the performance of the proposed method on this anisotropic reconstruction is slightly weak, compared with the three isotropic reconstructions aforementioned. This is primarily due to this material is more topologically complicated in pore s lineal size and morphology. Additionally, notice that here no explicit prior information associated with direction and size of pore is incorporated during reconstruction, and based on the loss functions used, our method can still capture and reproduce the essential directional morphology and dimensional characteristics. Of course, more constraints can be incorporated into our method to further improve the reconstruction accuracy.

V Conclusion

Reconstruction of porous media from limited information has been an outstanding challenge, especially the given data is scarce. To address this problem, in this paper we have presented a deep learning-based method for reconstructing full images of porous media from its much smaller sub-area(s), which provides a framework to produce accurate, fast, and robust realizations. Especially, CGAN is employed to learn the mapping between partial image and full image. In particular, two object functions are proposed and along with other two constraint functions, then jointly constitute the total object function to ensure the reconstruction accuracy. Besides, a Gaussian noise is introduced to preserve reconstruction diversity, allowing for multiple choices for users. Extensively tested on various porous materials, our method has been demonstrated to be able to accurately and stably reconstruct statistically equivalent structures while keeping high efficiency. By using our approach, despite the variety of amount and form of given data, the mapping between input and output can be successfully learned, and consequently the corresponding full images could be instantaneously reproduced. This may be especially useful in the applications that data is lacking or data acquisition is costly.

We would like to highlight that the proposed framework can be readily extended to other applications, such as 2D to 3D or 3D to 3D conditional reconstruction. Theoretically, it is able to incorporate an arbitrary number of object functions of any types into reconstruction, and incorporate any user-define conditional data, which could be of particular use in practice when the presence of specific structures in the area is informed. In addition, it also has the potential to reduce computational cost, for instance, it can be coupled with a variety of reconstruction methods of porous media like MPS Okabe and Blunt (2005); Gao et al. (2015); Ding et al. (2018), DS Mariethoz et al. (2010), CCSIM Tahmasebi and Sahimi (2012, 2013, 2016a, 2016b); Tahmasebi (2017), and used for accelerating the matching process in these algorithms. Some of which will be reported in the near future.

Acknowledgements.

This work is supported by the National Natural Science Foundation of China (Grant No. $61372174$). The authors would like to thank Dr. Daniel W. Apley, Department of Industrial Engineering and Management Sciences, Northwestern University, for providing several images of porous media used in this work.