NNVA: Neural Network Assisted Visual Analysis of Yeast Cell Polarization Simulation

TL;DR

This paper introduces a neural network-based visual analysis system that enables efficient exploration and calibration of high-dimensional input parameters for complex yeast cell polarization simulations, reducing the need for costly simulations.

Contribution

The work presents a novel neural network surrogate model integrated into a visual analysis system for interactive parameter calibration and sensitivity analysis of expensive biological simulations.

Findings

Neural network surrogate effectively approximates the yeast cell polarization simulation.

System enables real-time parameter sensitivity analysis at multiple levels of detail.

Provides insights into the simulation model through neural network interpretability techniques.

Abstract

Complex computational models are often designed to simulate real-world physical phenomena in many scientific disciplines. However, these simulation models tend to be computationally very expensive and involve a large number of simulation input parameters which need to be analyzed and properly calibrated before the models can be applied for real scientific studies. We propose a visual analysis system to facilitate interactive exploratory analysis of high-dimensional input parameter space for a complex yeast cell polarization simulation. The proposed system can assist the computational biologists, who designed the simulation model, to visually calibrate the input parameters by modifying the parameter values and immediately visualizing the predicted simulation outcome without having the need to run the original expensive simulation for every instance. Our proposed visual analysis system is driven by a trained neural network-based surrogate model as the backend analysis framework. Surrogate models are widely used in the field of simulation sciences to efficiently analyze computationally expensive simulation models. In this work, we demonstrate the advantage of using neural networks as surrogate models for visual analysis by incorporating some of the recent advances in the field of uncertainty quantification, interpretability and explainability of neural network-based models. We utilize the trained network to perform interactive parameter sensitivity analysis of the original simulation at multiple levels-of-detail as well as recommend optimal parameter configurations using the activation maximization framework of neural networks. We also facilitate detail analysis of the trained network to extract useful insights about the simulation model, learned by the network, during the training process.

Equations

$$PF=\Big{(}1-2\frac{S_{p}(C42a)}{SA}\Big{)}\times\frac{(ax)^{5}}{1+(ax)^{5}}$$

$$\underset{\text{x}}{\max }{f}_i(\text{x})-\lambda ||\text{x}-{\text{x}}^{\prime }|{|}^2$$

Full text

\CCScatlist\CCScat

K.6.1Management of Computing and Information SystemsProject and People ManagementLife Cycle; \CCScatK.7.mThe Computing ProfessionMiscellaneousEthics

\teaser

Approach Overview: A trained neural network-based surrogate model acts as the backend analysis framework, driving our interactive visual analysis system for analyzing a computationally expensive yeast simulation model.

\vgtcinsertpkg

Introduction

In the field of computational biology, scientists often design mathematical simulation models to offer quantitative descriptions of complex biological processes. These simulations are subsequently used to perform in-depth analyses of the real biological phenomenon. However, designing an optimal simulation model can be challenging. Scientists need to have a clear picture of how the different simulation input parameters are affecting the simulation output. For compute-intensive simulation models with high-dimensional input and output spaces, this can become a computationally prohibitive and non-trivial analysis task.

We collaborated with computational biologists to design an interactive visual analysis framework, which can assist them in analyzing and visualizing a complex yeast cell polarization simulation model. The model simulates the concentration of important protein molecules along the membrane of a yeast cell (single-cell microorganism) during its mating process. Cell polarization refers to asymmetric localization of protein concentration in a small region of the cell membrane and is a fundamental stage in the life-cycle of many microorganisms. Our experts are interested in exploring and analyzing the simulation input parameters, which can simulate varying levels of cell polarization results, particularly, the ones with high polarization. However, there are 35 different unknown/uncalibrated input parameters for the simulation. Besides the high-dimensional nature of the problem, the simulation model itself is computationally expensive. It takes hours on a supercomputing cluster to complete a single execution of the model. This seriously hampers the possibility of performing any exploratory analysis task that requires frequent execution of the simulation on new and unseen parameter configurations to study its properties in detail. In the field of simulation sciences, a popular and effective strategy to address this issue has been to create a simpler statistical/mathematical surrogate model, mimicking the original expensive simulation model [46, 14, 17, 47, 23]. The surrogate is then utilized to perform detailed analysis tasks instead of the expensive simulation model. A well-trained surrogate model can greatly facilitate the analysis workflow of complex simulation models.

Compared to popular surrogate model options like polynomial fitting or Gaussian Processes, neural networks are particularly well-suited for designing interactive visual analysis systems. This is primarily because, besides accurately predicting the output of high-dimensional non-linear functions, they can also be utilized to extract and analyze interesting properties about the original simulation by opening up the black-box of the trained neural networks. Recent advances in the field of interpretability and explainability of neural network-based models [39] have resulted in many useful post-hoc analysis techniques, making them more transparent in the process. This has led to a surge in their usage as proper analysis tools in many application domains [4, 26, 54, 50]. In this work, we propose a n**eural network-assisted v**isual analysis system (NNVA), which utilizes a neural network-based surrogate model to perform exploratory analysis and visualization of the aforementioned yeast simulation model. The surrogate model acts as the backend analysis framework, facilitating various visual interactions and analysis activities in the system. Of late, there is a growing interest in the usage of different neural network-based models to solve complex real-world problems. Our work, therefore, exemplifies how to build an interactive visual analysis system around these powerful models, i.e, Machine Learning for Visual Analytics (ML4VA).

Our proposed system facilitates interactive exploratory analysis by allowing the experts to modify the input parameter values and immediately visualize the predicted simulation outcome. This helps them discover new parameter configurations without having the need to execute the original expensive simulation for every instance. Using different interactive selection brushes, experts can perform parameter sensitivity analysis at multiple levels of detail as well as get optimal parameter recommendations to produce desired simulation outcomes for selected regions of the membrane. To establish the trustworthiness of any visual analysis system, it is important to convey the underlying uncertainty associated with the predictions of the surrogate model. We utilize a recently proposed uncertainty quantification technique for neural networks using dropout layers [15] to incorporate uncertainty visualization in our system. We also allow the experts to validate the surrogate model itself, by analyzing the various weight matrices and extracting the knowledge learned by the surrogate during the training process. We performed extensive evaluations of the proposed framework by comparing with the original simulation model and the results of a previous analysis effort which used polynomial surrogate models [48].

To summarize, the major contributions of our work are as follows:

• •

We demonstrate how a trained neural network can act as an analysis backend to drive an interactive visual analysis system.

• •

We discovered multiple previously unknown parameter configurations, which generated strong cell polarization results in the original simulation model.

• •

We provide easy integration of our visual analysis workflow with the simulation modeling workflow of the experts by allowing them to store the discovered configurations in a file format, which can be used to directly execute the original simulation model.

1 Related Work

In this section, we focus on previous research works from the field of visual analysis, which are relevant to our proposed approach.

Visual Exploration of Parameter Space: Over the years, multiple visual analysis systems have been proposed to facilitate interactive visual exploration of the input parameter space for simulation models. Each of them are application specific, and caters to the requirements of their domain experts. Orban et al. [44] projected input parameters and output data in material science to 2D spaces, and allowed users to manipulate in the input space and observe the change in the output space. Wang et al. [60] developed a nested parallel coordinate plot for parameter analysis of multi-resolution climate ensemble datasets. Biswas et al. [7] used a Gaussian Process-based surrogate model to perform interactive exploration in a shock physic application. Coffey et al. [9] designed an interface which uses a mapping between model features and simulation inputs to enable direct simulation input parameter manipulations. Piringer et al. [45] proposed an approach called HyperMoVal, which can evaluate the bad fit of surrogate models and provide visual validation for their physical plausibility. Berger et al. [5] proposed an uncertainty-aware statistical approach to predict results of given parameters for real-time analysis.

Sedlmair et al. [51] provided an extensive survey and proposed a conceptual framework to categorize the various analysis tasks and navigation strategies used in such visual analysis systems. Our proposed visual analysis system encompasses three of the six analysis tasks formalized by Sedlmair et al., namely, optimization, uncertainty, and sensitivity. Among the four navigation strategies that they identified, our system covers two of them, namely informed trial and error and local-to-global. Simulation parameter analysis is also a popular topic in scientific visualization community. Parameter sensitivity analysis techniques [41, 28, 12] have been widely used to perform various uncertainty-aware scientific analysis and visualization [8, 18, 64]. The recent survey on visualization techniques for ensemble simulation data by Wang et al. [59] also covers the sub-category of simulation parameter analysis in the visualization community.

Machine Learning Models for Visual Analysis: Visualization community often uses machine learning techniques to enhance their visual analytic tools [13]. Machine learning models act as a medium to extract interesting insights about the data, which is then presented to the end users through interactive visual analytic systems. Besides enhancing the data-analysis experience, this acts as a platform for users without much machine learning background to reap the benefits of sophisticated machine learning models. Among the recent neural network-based models, CNN (Convolutional Neural Network) [63] and Word2Vec [66, 36] models have been used to create interactive visual analysis systems for different application domains. Moreover, traditional models like SVM (Support Vector Machine) [62], LDA (Latent Dirichlet Allocation) [31, 35], KNN (K Nearest Neighbor) [37], Bayes’ rule [16], learning-from-crowds model [34], and online metric learning [32] have also been extensively utilized by visualization researchers to enhance the data-analysis experience in their systems. Along similar lines, our proposed system utilizes a trained multilayer perceptron model to design an interactive visual analysis framework for a scientific application.

Visual Analysis for Machine Learning Models: Since the past few years, the visualization community has played a significant role in explaining the inner workings of complicated machine learning models. Multiple visual analytic tools have been developed to visualize different machine learning algorithms, such as Adaboost, SVM, decision tree, and random forest [22, 20, 10, 49, 55]. Recently, Hohman et al. [21] published a comprehensive survey on the various visual analytic approaches to explain deep neural network models, which are gaining significant popularity in the machine learning community. At a high-level, we can divide these approaches into three categories. The goal of one category of visualization tools is to open the black box by interpreting the trained model [61, 33, 38, 57, 53, 25]. While, another category of visualization tools not only interpret, but also diagnose the trained model [24, 65, 52, 58]. Recently, a third category of visualization tools, focusing more on assisting the machine learning experts to improve their models is gaining popularity[56, 6].

Besides the visualization community, the machine learning community is also working in parallel to create various post-hoc analysis techniques to interpret and explain complicated models. Unlike most of the visualization tools, these post-hoc analysis functions are intended to be more generic and applicable for different network architectures. Montavon et al. [39] covers in great details the various post-hoc analysis techniques that can be performed on trained neural networks to make them more interpretable and explainable. These analysis techniques for trained neural networks have seen wide-spread application in scientific domains, ranging from cancer diagnosis to quantum physics [4, 26, 54, 50]. In our proposed visual analysis system, we use different post-hoc analysis functions on the trained neural network-based surrogate model to study and analyze the original yeast simulation. We also visualize the network structure (weight matrices) to extract and validate the knowledge learned by the surrogate model during the training process.

2 Simulation Model Background

Traditional laboratory-based approach for studying yeast cell polarization consists of a laborious workflow. The cells are first cultured/grown in a growth media and then treated with various straining agents. They are then visualized using high-resolution microscopes. Fig. 1(a) shows the microscopic image of a highly polarized yeast cell. A mathematical simulation model, therefore, can significantly accelerate the study of such biological phenomena.

To capture the spatio-temporal dynamics of yeast cell polarization during the mating cycle, scientists created a mechanistic spatial model to simulate the concentration of important protein species along the cell membrane. In this work, we focus only on one important protein species called Cdc42. Protein concentration is measured as the number of molecules per unit area of the membrane. The model simulates the cell membrane as a circle, centered at the origin with radius $2\mu m$. Fig. 1(b) shows a pedagogical diagram of the yeast cell structure with the peripheral cell membrane, while, Fig. 1(c) shows the corresponding computational domain of the simulation model used by the scientists. The computational domain (circle) has a spatial resolution of 400, parameterized by angles in the range $[0\degree,360\degree]$. Scientists are interested in simulating results with high degree of polarization, in particular of Cdc42 protein. To quantify the extent of Cdc42 polarization, they constructed a scalar function of active Cdc42 (C42a) values called polarization factor (PF), denoted as;

[TABLE]

where, $SA$ is the surface area of the membrane simulated by the model, $S_{p}(C42a)$ is the surface area at the front of the cell that encompasses half of the polarized component C42a, $x$ is the maximum C42a concentration value and $a$ is an experiment constant dependent on simulation parameter. An unpolarized cell would have a PF value of 0 and an infinitely polarized cell would have a PF value of 1.

However, the simulation model comprises of 35 different input parameters. For the model to be useful to study the biological process of yeast cell polarization, scientists need to have a clear understanding of how the simulation input parameters effect the simulation results. More specifically, they want to figure out the parameter configurations which can generate high cell polarization results in the model. Studying this high-dimensional parameter space is not trivial, especially when the individual simulation execution itself takes few hours to execute. Any analysis task that requires frequent execution of the model on new parameter configurations is effected by the long execution time of the simulation model.

Previous Simulation Model Analysis: Previous efforts into analyzing this simulation model involved creating a polynomial surrogate model [48]. The surrogate model was created by uniformly sampling the parameter space and fitting a polynomial function to the polarization factor (PF) values (Equation 1). The surrogate model facilitated in analyzing the parameter sensitivity of the model and helped estimate parameter configurations using a Markov Chain Monte-Carlo (MCMC) approach. However, the approach was not able to identify satisfactory parameter configurations with which the simulation can generate significantly high polarization results. Moreover, given a parameter configuration, the polynomial surrogate model only predicts the final PF value and not the Cdc42 protein concentration values across the membrane, which is the final output of the simulation. As a result, parameter sensitivity analysis performed with the polynomial surrogate model was not able to study the influence of the parameters on different regions of the membrane in finer details. Such analysis is important to get a better understanding of how the simulation parameters actually affect the protein concentration across the membrane and not just the final PF value. In this work, we train a neural network-based surrogate model to predict Cdc42 concentration values for the 400 spatial locations across the membrane as modeled by the computational domain (Fig. 1(c)). We use this neural network-based surrogate model to create a visual analysis system for the simulation model. We utilize some of the important findings in the previous work [48] to extensively evaluate and validate the results produced by our proposed system.

3 Requirement Analysis and Approach Overview

Requirements: Throughout the course of this project, we had multiple interactions with the scientists from computational biology to understand the various aspects of their simulation model and get a clear picture of their needs and requirements from a visual analysis system. Based on these discussions, the most important requirements are as follows:

• R1

Discover new parameter configurations which can generate high Cdc42 polarization results in the simulation model. The system should have the ability to visually guide the users in the process of finding desired parameter configurations for the simulation.

• R2

Ability to get a quick preview of the predicted simulation output for particular parameter configuration to facilitate model calibration. This helps them decide whether to execute the expensive simulation model with certain parameter configurations or not.

• R3

Perform detailed sensitivity analysis of the input parameters with respect to Cdc42 concentration for different regions of the cell membrane.

• R4

Analyze the distribution of protein concentration values across the computational domain of the model. This is required to decide on an ideal partitioning scheme for the computational domain.

• R5

Ability to extract and validate the knowledge learned by the surrogate during its training process. This is required to make sure that the trained network is not making any random predictions.

Overview: Based on these requirements, we have proposed an interactive visual analysis system, which is driven by a neural network-based surrogate model. We first train a fully connected neural network on a finite set of training data, obtained by running the simulation on random parameter configurations. The neural network learns to predict the concentration of the Cdc42 protein along the cell membrane for a given input parameter configurations. We then design a visual analysis framework which allows the users to visually query for different properties about the simulation model to address the aforementioned user requirements. These queries are executed in the backend by the trained surrogate model to provide prompt feedback via the visual interface. The system visually guides the users to discover new parameter configurations, which can be later used to execute the original simulation model. Fig. NNVA: Neural Network Assisted Visual Analysis of Yeast Cell Polarization Simulation shows the high-level overview of the proposed system.

4 Neural Network-based Surrogate Model

Surrogate models are widely used in many areas of engineering and simulation science as cost effective alternatives to expensive simulation models for various analyses [46, 14, 17, 47, 23]. Also known as response surface models or emulators, they mimic the behavior of the actual simulation model as closely as possible while being computationally easier to evaluate. The surrogate model proposed in our work is a multi-layer fully-connected feed-forward regression neural network. In this section, we first discuss in details the network structure and the training process of our surrogate model and then elaborate on the various post-hoc analysis techniques that can be performed on trained networks to facilitate the eventual visual analysis system.

4.1 Network Structure and Training Process

As shown in Fig. 2(a), our surrogate model consists of 5 layers, comprising an input layer of 35 neurons and an output layer of 400 neurons, corresponding to the 35 simulation input parameters and 400 uniformly distributed spatial locations along the cell membrane respectively. The intermediate hidden layers, $H_{0}$, $H_{1}$ and $H_{2}$ are composed of 1024, 800 and 500 neurons respectively. The network is fully-connected, therefore, the neurons of one layer are connected with all the neurons of its subsequent layer. The output of each neuron in the three hidden layers are passed through ReLU (rectified linear unit) activation functions [42] to model any non-linearity between the input parameters and the output responses. We also apply dropout regularization to the first ($H_{0}$) and second ($H_{1}$) hidden layers with a dropout rate of 0.3. Dropout regularization corresponds to randomly ignoring the activation results of the neurons in a layer during the training process to avoid over-fitted networks and achieve higher accuracy. A dropout rate of 0.3 refers to the fact that we randomly ignore $30\%$ of the neuron outputs at layers $H_{0}$ and $H_{1}$.

We trained the surrogate model on a training dataset of size 3000. Our experts predetermined the ranges of the individual parameters and normalized them independently to the range $[-1,1]$. The training data was created by first randomly sampling the 35 parameters in their normalized value ranges to create 3000 random parameters configurations and then running the simulation model to get the corresponding Cdc42 concentration values for each configuration ($\sim 27$ hours on a supercomputing cluster). It was observed that a large fraction of the randomly sampled training data corresponds to instances with very low Cdc42 polarization. Therefore, to train the network to predict the high polarization instances (R1), we performed PF value weighted training of the neural network. During the training iterations, this assigned high weightage to the loss function values corresponding to training data instances with high PF values. After training for 5000 epochs, with a batch size of 32, the model achieved a stable RMSE (root mean square error) accuracy of $87.6\%$. Standard mean squared error (MSE) was used as the loss function for training the network and the accuracy was tested on a separate validation dataset of size 500. We tested different network architectures before finalizing on the one described in Fig. 2(a) because it had the highest accuracy and stabilized relatively faster (i.e, around $3500^{th}$ epoch). The complete accuracy profile during the training process is provided in the supplementary materials.

4.2 Uncertainty Quantification in Neural Network

Traditionally, neural networks do not provide any measure of the uncertainty associated with its predictions by default. However, in a recent work, Gal et al. [15] showed that traditional neural networks can also be made to quantify uncertainty by activating the dropout layers in the prediction (testing) phase. As discussed in Section 5.1 above, dropout layers are generally used as regularizers in the training phase to avoid over-fitting of training data by randomly ignoring the neuron activations at different layers of the network. Dropout is generally turned off when the network makes predictions. Gal et al. [15] showed that if we apply dropout during prediction and randomly ignore the activations of neurons in different layers, we get slightly varying predictions every time the network is run with the same set of input. By observing the variations of the predicted results for multiple instances, we can quantify the uncertainty associated with the predicted results. Gal et al. [15] further proved that dropout induced uncertainty for neural networks is actually an approximation of the uncertainty obtained in Bayesian models like Gaussian Processes. Therefore, using this feature in our trained neural network-based surrogate model, we can incorporate uncertainty visualization into our proposed visual analysis system.

Fig. 2(b) demonstrates the uncertainty visualization results using dropout layers in a simple 3-layer neural network on a synthetic dataset. Consider using a neural network to learn a simple sinusoidal function $f(x)=sin(2\pi x)-x$. Training data to learn this function was provided only for the $x$ ranges of $[-2.0,-1.5]$ and $[-1.0,0.5]$. Fig. 2(b1) shows the plot of the true function values $y=f(x)$ for the training data ranges. Fig. 2(b2) shows the result of the trained neural network predictions for $x$ values ranging from $-2.0$ to $3.0$. The variation in the predicted result is captured using dropout layers and is visualized as standard deviation bands (in shades of green) around the mean predicted values. As can be seen, the uncertainty is high for regions where the training data was not provided to the network to learn from. The uncertainty visualization clearly shows that as we move away from the training data range (i.e, $x>0.5$) the corresponding prediction uncertainty also increases. In order to avoid misleading the users in their decision making process and to add a sense of trustworthiness, it is vital for a visual analysis system to convey any underlying uncertainty in the model.

4.3 Parameter Sensitivity Analysis

Sensitivity analysis is a popular post-hoc analysis technique performed on trained neural network models to identify the most important/salient input features. It serves as an effective tool in driving many recent advances in the field of explainable machine learning [39].

Sensitivity analysis of a neural network corresponds to computing the partial derivative of the outputs with respect to the inputs. Consider the $i$-th neuron in the output layer, predicting the function $f_{i}(\textbf{x})$ for an $n$-dimensional input vector $\textbf{x}\sim\{x_{1},...,x_{n}\}\in\mathbb{R}^{n}$. The sensitivity of $f_{i}$ with respect to the $j$-th input parameter can be denoted as \Big{(}\frac{\partial f_{i}}{\partial x_{j}}\Big{)}^{2}. A high sensitivity value corresponds to the fact that a small change in the value of the input $x_{j}$ is going to have a significant change in the output value of $f_{i}(.)$. The architecture of neural network is such that the output of every neuron in the network is completely differentiable with respect to its inputs, as a result, we can easily compute the required partial derivatives for sensitivity analysis via chain-rule using the backpropagation technique [19]. In our work, we utilize this to evaluate the sensitivity of the 35 simulation input parameters using the trained surrogate model. The visual analysis system provides an interface for the scientists to query for such parameter sensitivity information for different spatial regions of interest along the cell membrane. Another advantage of using neural network for sensitivity analysis is that we can also compute the sensitivity of the hidden layer activation values with respect to the input parameters as well. We utilize this to observe the parameter sensitivity towards interesting latent space data patterns learned by the hidden layers of the surrogate model during the training process.

4.4 Parameter Optimization

Another important category of post-hoc analysis operation performed on trained neural networks is called activation maximization (AM). Activation maximization corresponds to searching for an optimal input configuration in the high-dimensional input space that maximizes the output response function. It is often used to interpret a high-level concept learned by the neural network, for example, in the field of image classification, it can be used to create new images of what the trained network thinks a cat or a dog looks like [43, 39].

Neural networks are essentially optimization machines that try to find the optimal network configurations (various weight and bias values) during the training process that can best map a given input to the desired output. Once the training process is over, the network configuration is fixed and is used to predict outputs for new and unseen input configurations. Activation maximization corresponds to a reverse optimization process, where, keeping the network configuration fixed, we search for optimal input configurations that maximizes the function values of specific neurons in the output layer. For the $i$-th neuron in the output layer predicting the function $f_{i}(\textbf{x})$ for an $n$-dimensional input vector $\textbf{x}\in\mathbb{R}^{n}$, we can find an optimal input configuration $\textbf{x}^{*}$ by optimizing the following objective function

[TABLE]

where, the rightmost term is an $\ell_{2}$-norm regularizer to constrain the input search space within a known confinement $\textbf{x}^{\prime}$. This penalizes the optimizer from finding an arbitrarily different $\textbf{x}^{*}$. The optimization involves a gradient ascent algorithm using the gradients $\frac{\partial f_{i}}{\partial x}$ to update the inputs to eventually find the optimal input configuration. A similar approach can also be employed to minimize the activation of a selected neuron, i.e, activation minimization, by negating the gradient values during the optimization steps. In our work, we utilize activation maximization and minimization principles to recommend simulations input parameter configurations to the scientists. They can visually selected the spatial regions in the cell membrane that they want to maximize/minimize the Cdc42 protein concentration for. By carefully choosing to maximize Cdc42 concentration in certain regions and minimize in the other regions scientists can query for parameter configurations that is likely to produce high polarization profiles in the original simulation.

5 Neural Network Assisted Visual Analysis

In this section, we introduce our proposed interactive visual analysis system. We first explain the primary visualizations and interaction techniques before looking at the high-level analysis views in our system.

5.1 Primary Visualizations and Interactions

Cdc42 Visualization: To visualize the predicted Cdc42 concentration and preserve the circular context of the cell membrane structure, we opted for radial layout designs [11]. Fig. 3(a) visualizes the mean predicted concentration values for the 400 uniformly sampled points across the membrane. By default, the values are color-mapped to the maximum and minimum Cdc42 values observed in the simulation by the experts. Radial coordinate axes (dashed-gray lines) are provided in the backdrop as frame of reference to reflect the parameterization of the simulation domain in terms of angles (degree). To efficiently utilize the design space, we employed the popular design principle of superposition, i.e. visualizing multiple data subsets in the same coordinate system [40].

Uncertainty Visualization: The uncertainty associated with the predicted values of the surrogate model is visualized using superimposed standard deviation bands around the circumference of the circular domain. The shapes of first (inner) and second (outer) standard deviation bands, as shown in Fig 3(a), highlight the deviation of the predicted values in the corresponding locations of the membrane.

Selection Brushes and Interaction: As shown in Fig. 3(a), we superimpose multiple interactive radial selection brushes in the same coordinate system as the Cdc42 visualization. This facilitates performing various visual queries on different regions of the membrane. The sensitivity brush (gray) allows the users to select the regions of the membrane where they want to perform detailed parameter sensitivity analysis. This brush is linked with the sensitivity visualizations in Fig. 3(c) and (f). The maximization brush (red), selects the region where the users want to maximize the predicted concentration values (Section 5.4). On clicking this brush, the corresponding optimal parameter values, computed by the backend surrogate model, are reported in the parameter control bars (Fig. 3(b)). Similarly, the minimization brush (blue) selects the region to minimize the predicted values. The green circular button at the center performs a logical AND operation of the regions selected by the maximization and minimization brushes.

Parameter Control Bars: Fig. 3(b) shows the parameter control bar for one of the simulation input parameters. The parameter name is followed by an input textbox to enter desired parameter values. The sliderbar, immediately below, can also be used to adjust the parameter values. The parameter values corresponding to different configuration instances show up in the last bar. The value corresponding to currently loaded instance show up in black colored text, whereas, the optimal parameter values recommended by activation maximization, minimization and combined max-min show up as red, blue, and green colored texts respectively. Users can click on these texts to adjust the parameter value as well.

Sensitivity Visualizations: For a given parameter configuration, the local sensitivity of the 400 spatial locations across the membrane with respect to the 35 parameters are visualized in a circular heatmap as shown in Fig. 3(c). The 35 parameters are laid out along the radial direction. A high sensitivity score for a parameter implies that a small change in its current value is going to trigger a relatively high change in the predicted Cdc42 value for the specific region of the membrane. Finer spatial selections can be made using the aforementioned sensitivity brush. The average sensitivity of the 35 different parameters across all the spatial locations is shown as a bar-chart in Fig. 3(e). The average parameter sensitivity information for a user selected region of interest (via sensitivity brush) is shown in Fig. 3(f), juxtaposed with (e) to convey the relative variation in the sensitivities.

Cluster Visualization: To help analyze the current partitioning scheme of the simulation domain (R4), we perform hierarchical clustering [30] of the 400 uniformly partitioned points based on their Cdc42 values and associated uncertainty. The multi-level cluster information is visualized using a radial dendogram as shown in Fig. 3(g). Hovering over the nodes highlights the corresponding selected clusters in the simulation domain (Fig. 3(g1)). Similar clustering is performed on the average sensitivities of the 35 parameter to study their associations. Fig. 3(d) shows a linear dendogram view of the parameter cluster information. Users can control the tree depth, which relates to the number of clusters desired for the study. For selected nodes in the tree, the corresponding cluster members get highlighted in Fig. 3(e) and (f).

Weight Matrix Visualization: To extract the knowledge learned by the trained neural network, and thereby, validate the surrogate model (R5), we present various techniques to analyze its weight matrices. Fig. 3(h) shows the weight matrix ($35\times 1024$) between the input layer and the first hidden layer (H0). The rows correspond to the 35 input parameters and the columns correspond to the final weights assigned to the 1024 neurons in H0 layer. On clicking the matrix, the weights are sorted in ascending order for each row of the matrix, which helps identify interesting weight distribution patterns as shown in Fig. 3(h1). Detailed explanation about the interesting weight patterns and their importance is provided in Section 7.2. On hovering the mouse over the rows, we display the shape of the corresponding weight distribution for the respective parameter in a pop-up window (Fig. 3(h2)). Based on the patterns observed in the final weight matrix ($500\times 400$) between the last hidden layer (H2) and the output layer, as shown in Fig. 3(i), we offer a separate set of interactions to study the matrix. We provide a row selection probe in the form of a sliderbar with an arrowhead (Fig. 3(j)) to select the rows with interesting weight patterns. The selected row index is highlighted in the side and a zoomed-in view of the row is displayed (Fig. 3(i1)). Fig. 3(k) shows the average parameter sensitivity chart corresponding to the neuron in the penultimate layer (H2), which is responsible for the selected weight pattern.

Colormap Adjustments: We offer two sets of value ranges for mappings the colors when visualizing the predicted Cdc42 concentration. The first set (default) is based on the minimum and maximum concentration values that the experts feel is effective for studying cell polarization behavior. For the current system, this range is set to $[0,400]$. The second set corresponds to the local minimum and maximum values for individual prediction results. This provides more control to the experts during the analysis period, because the concentration values for different instances can have varying dynamic ranges. We have included nine divergent and three sequential colormaps [3] in our system for the users to choose from.

5.2 Visual Analysis System

Using the visualizations and interaction techniques explained above, we design our visual analysis system with multiple high-level views to provide a structured and efficient analysis workflow for our experts. It comprises of the following high-level views, each addressing different facets of the analysis requirements set forth in Section 4.

Instance View: This view loads the visualizations corresponding to a specific instance of input parameter configuration that the scientists wish to analyze in detail (R3, R4). As shown in Fig. 4(a), it comprises of two sub-views. Cdc42 Viz displays the predicted Cdc42 concentration result for the currently analyzed parameter instance, whereas, Parameter Sensitivity View displays the corresponding spatial sensitivity as well as the overall average parameter sensitivity for the specific instance. Both the sub-views have the corresponding cluster trees described in Section 6.1 for detail analysis. In Cdc42 Viz, there is a switch to toggle between the radial cluster trees corresponding to the Cdc42 values and the uncertainty values (R4).

Parameter Control Board: This view serves as the main panel to visualize and interactively modify the input parameter configurations (R1). As shown in Fig. 4(c), the 35 different parameter control bars (Section 6.1) for the individual parameters are laid out across four rows. It visualizes the parameter values corresponding to the parameter instance currently analyzed in the Instance View as well as the optimal parameters recommended by the interactive optimization brushes. Users can modify the parameter values and click the Run button to execute the neural network-based surrogate model in the backend to generate the corresponding simulation prediction, which is visualized in the Quick View (R2). The Save button lets us store the modified input configuration in the Parameter List View, whereas, the Export button downloads the list of saved configurations. Instead of manually adjusting the 35 different parameter sliders, users can click the Current, Maximize, Minimize or MaxMin buttons to automatically set the parameter sliders/values to the desired recommended configurations.

Quick View: As shown in Fig. 4(b), it visualizes the predicted Cdc42 concentration for the user-modified parameter configurations via the Parameter Control Board. This offers a means to perform rapid prototyping of the expensive simulation using the surrogate model, thus facilitating exploratory analysis with new and unseen parameter configurations (R1, R2). In order to compare the predicted Cdc42 values vis-à-vis the results in the Instance View, we provide a toggle switch to change the visualization in Quick View to display the exact difference in Cdc42 concentration across the membrane.

Parameter List View: As shown in Fig. 4(d), this view temporarily stores the newly discovered parameter configurations. Additionally, users can click on the rows in the list to load the selected configuration back in the Parameter Control Board. This list of configurations can be exported/downloaded in a file format which can be directly used to execute the original simulation model. This offers a seamless integration between the analysis workflow involving our visual analysis system and the actual simulation modeling workflow of the experts (R1).

Model Analysis View: This view lets the users investigate the trained neural network-based surrogate model to extract useful insights about the simulation model (R5). As shown in Fig. 4(e), the left-most panel shows the thumbnail views of all the weight matrices of the trained network. Users can click on the matrix images to open up the corresponding analysis views in the right panel. Detailed analysis, as explained in Section 6.1, can be performed on the selected weight matrix to extract any data patterns and validate the knowledge learned by the surrogate model.

6 Case Study and Evaluation

In this section, we perform two case studies using our visual analysis system and evaluate the results by comparing against the original simulation outcomes as well as the findings from a previous polynomial surrogate model based analysis of the simulation [48].

6.1 Discover New Parameter Configurations

One of the key requirements from the system is to visually guide the users towards discovering desired parameter configurations (R1), instead of having to perform random sampling of the high-dimensional parameter space. This also enables the experts to incorporate their domain knowledge into the parameter discovery process. In this case study, we use our system to identify new parameter configurations that can trigger high Cdc42 polarization in the original simulation model. We support two different visual parameter discovery approaches. According to the conceptual framework of Sedlmair et al. [51], both of these approaches can be categorized under local-to-global and informed trial and error visual parameter navigation strategies.

Guided by the Instance View: In this approach, users can center their parameter discovery process around a known instance of input parameter configuration, whose results get loaded in the Instance View. The Instance View offers in-depth analysis of the Cdc42 concentration as well as the corresponding parameter sensitivity information for the loaded parameter instance. Fig. 5(a), (b) and (c) show the zoomed-in views of the predicted protein concentration, spatial parameter sensitivity and average parameter sensitivity respectively of a selected region for the loaded parameter instance. As can be seen in Fig. 5(a), the loaded instance corresponds to a good polarization profile. Using this as the starting point, users can start modifying the parameter configurations based on the parameter sensitivity details of the loaded instance combined with their domain knowledge.

For example, from the sensitivity views in Fig. 5(b) and (c), we can infer that the parameters k_42a and k_42d are the most sensitive parameters for the loaded configuration. This implies that a small change to these parameter values will significantly change the current polarization profile. We use the Quick View panel to visualize the predicted polarization profile generated by the modified parameter values. Fig. 5(d) shows the results for decreasing and increasing the current k_42a value ($-0.24$) by a step-size of $0.05$ in the top and bottom images respectively. Decreasing the parameter value significantly brings down the protein concentration at the top of the cell, while, increasing the parameter value increases the concentration. A reverse trend is observed for k_42d (Fig. 5(e)), where increasing the parameter values by $0.05$ reduces the concentration and vice-versa.

From the simulation perspective, this behavior makes sense because k_42a corresponds to Cdc42 activation, whereas, k_42d corresponds to Cdc42 deactivation. Since the type of simulated protein is active Cdc42 (i.e, C42a), k_42a have a positive impact on its concentration while k_42d have a negative impact. Similarly, for a less sensitive parameter like k_RL, we can see in Fig. 5(f) that changing its parameter value by the same step-size does not result in a significant change in the polarization profile. Using this approach, we identified 8 new parameter configurations which are likely to produce high Cdc42 polarization results in the original simulation.

Guided by Parameter Optimization Brushes: We also recommend new parameter configurations based on the activation maximization/minimization analysis framework of neural network described in Section 5.4. Users can utilize the maximization brush, as shown in Fig. 5(g), to select the region of the cell membrane where they want to maximize the predicted Cdc42 value. The corresponding optimal parameter configurations computed by the surrogate is recommended in the Parameter Control Board. Fig. 5(g1) shows the result of running the surrogate model with the recommended parameters. As can be seen in Fig. 5(g1), the predicted Cdc42 values in the selected region sharply increases (even beyond the maximum value of 400, set by the experts). Similarly, a minimization brush, as shown in Fig. 5(h), recommends a parameter configuration that brings down the concentration values of the selected region close to the minimum concentration value (Fig. 5(h1)). For the particular case of finding high polarization profiles, we are interested in maximizing the concentration at the top of the cell and minimizing the concentration across rest of the locations at the same time. We make the desired selection using the two brushes to make this query as shown in Fig. 5(i) and (j) for two different selection ranges. The corresponding recommended parameters display high Cdc42 polarization predictions, as indicated by the Quick View results in Fig. 5(i1) and (j1) respectively for the two different selections. On top of the recommended parameters, users can further modify individual parameter values to create new sets of parameter configurations. Using this approach, we created a total of 7 new input parameter configurations.

Evaluation: We executed the simulation model using the 15 newly identified parameter configurations and observed high degrees of Cdc42 polarization (PF $>0.5$) for all the configurations. We found 5 parameter configurations with PF values exceeding 0.8, whereas, in the initial training data that was collected by randomly sampling the parameter space, we never found an instance with PF value close to 0.8. The highest PF value recorded among the newly discovered parameters was 0.82 and corresponds to the optimized configuration recommended by the neural network for the selection shown in Fig. 5(j). In Fig. 6(a), we compare the actual simulation result generated with our discovered optimal parameter configuration versus using the parameters estimated by previous polynomial surrogate model based analysis [48]. The sharp blue curve (PF = 0.87) corresponds to the protein concentration obtained by extracting the pixel intensity values of a real microscopic image of a highly polarized yeast cell as shown in the blue box. This acts as the ground truth for the simulation model to generate similar levels of polarization results. The black plot shows the simulation result produced using the optimal parameters discovered by our system (PF = 0.82). The solid red plot (PF = 0.57) and dashed red plot (PF = 0.64) corresponds to the simulation results generated by the parameter configurations estimated in previous work [48].

This is a significant improvement over the previous parameter analysis results for the same simulation model. This establishes that, given the right input parameter setting, the simulation model is capable of generating sharp polarization results similar to real laboratory results. We also visually verified the predicted results of the surrogate model against the original simulation outputs by plotting them together as curves. Fig. 6(b) and (c) show the comparative plots for a highly uncertain prediction and a less uncertain prediction instance respectively. The green dashed-line shows the mean prediction curve, whereas, the solid black line is the original simulation output.

6.2 Knowledge Extraction from Surrogate Model

In this case study, we aim to extract the knowledge learned by the network during its training process by analyzing its various weight matrices. This also serves the purpose of validating the surrogate model to make sure that it is making predictions based on some reasonable domain-aligned logic rather than random ad-hoc predictions (R5).

First Weight Matrix: In the first weight matrix between the input layer and first hidden layer (Fig. 7(a)), we observe distinct patterns in the distribution of weights for certain parameters. Fig. 7(b) shows the full $35\times 1024$ matrix with the weight values sorted in ascending order for each row (i.e, parameter) to highlight the patterns (Fig. 3(h) shows the original matrix view). We observed relatively high negative and positive weights for parameters k_24cm0, k_24cm1, k_42a, k_42d, C42_t, q and h. High positive and negative weights in the first matrix for a parameter corresponds to the fact that the parameter values had to be scaled by the weights for some neurons in the first hidden layers. This implies that the original range of values provided for the parameters is not sufficient. Similar observations were made for these parameters explicitly in the previous analysis work [48]. The experts feel that the range of these parameters need to be expanded to get better simulation results. Besides, the matrix also verified that the pairs (k_24cm0, k_24cm1), (k_42a, k_42d), and (q, h) showed relatively strong correlation compared to the other parameters. The exact shape of the weight distributions for these parameters are provided in the supplementary materials.

Final Weight Matrix: We did not observe any distinct patterns in the second and the third weight matrices. However, in the final weight matrix of resolution $500\times 400$, corresponding to the third hidden layer and the output layer, we found multiple interesting weight distribution patterns (full matrix view is shown in Fig. 3(i)). Each row of this matrix corresponds to the weights assigned to individual neuron activation values in the penultimate layer ($H_{2}$) towards the 400 output neurons, as illustrated in Fig. 7(c). We observed that different neurons in $H_{2}$ layer assign high positive or high negative weights to different sets of output neurons. In the output layer, the neurons at the middle section corresponds to the top of the cell membrane modeled by the simulation. Therefore, it is interesting to identify which neurons in the $H_{2}$ layer assigns high positive weights to the middle section of the output layer, because those neurons are most likely to contribute towards producing high Cdc42 polarization results. We identified 95 such neurons in the $H_{2}$ layer (Fig. 7(d)) and evaluated their average parameter sensitivity (Section 5.3) to find out which parameters are more sensitive to produce the selected weight pattern in the penultimate layer.

Fig. 7(e) shows the sorted list of parameters based on descending order of their average normalized sensitivities. We compared this importance/sensitivity order of parameters for generating high polarization patterns with that of the list of highly sensitive parameters identified in the previous work of our experts [48]. We found that except the change in order of the 3 parameters q, D_c42a, and D_c42 by one position (marked by blue texts), the top 15 sensitive parameters (red texts) were in the same order as previously identified by experts. Similar analysis can be perform for different weight distribution patterns in the penultimate layer of the network.

7 Domain Expert Feedback

Our experts from the field of computational biology comprise of a professor from the Department of Mathematics, who created the yeast simulation model, and two of her graduate students. They feel that the proposed visual analysis system is a very useful tool to fine-tune their simulation model. They found the visual interface of our system to be simple and intuitive for users familiar with yeast simulation models. The ability to quickly prototype different parameter combinations and interactively visualize the predicted simulation output within seconds lets them easily calibrate the simulation model.

Previously, there was no interactive visualization created for the yeast simulation. Our experts feel that the visual analysis system will be a useful medium to communicate with the non-expert collaborators and stakeholders of the project, instead of explaining them the complex reaction-diffusion equations involved in the simulation. As discussed in Section 7.1, using our system we were able to discover new parameter configurations that can trigger high Cdc42 polarization in the original simulation model. This is a significant improvement over the previously estimated parameters using polynomial surrogate model analysis [48].

The experts feel that the system is flexible to work with other protein species besides Cdc42. The current backend, i.e, neural network-based surrogate model, predicting Cdc42 concentration, can be easily replaced with another neural network model predicting different species and still retain the same visual interactions to analyze the simulation. Since the backend analysis techniques are independent of the network structure, they can train a network with a different architecture and still utilize our visual analysis frontend. The experts also plan to utilize the radial clustering information to create an adaptive mesh for the computational domain rather than the current 400 uniformly spaced resolution.

The model analysis view was helpful to validate the trained network and see if the surrogate model is actually learning something relevant about the simulation rather than making random predictions. However, they feel that the model analysis view requires users to have a good understanding of the neural network architecture to interpret the weight matrices. They feel it would be helpful to make the weight matrix analysis more intuitive for people without much machine learning background. Overall, the experts are satisfied with our neural network assisted visual analysis system and feel that it meets all of their expected requirements. They plan to use the findings from the visual analysis system to improve the simulation and report in a systems biology journal in future.

8 Discussion

Design Choices: The choice of using radial layouts for some of the visualizations is inspired from the circular shape of the computational domain (Fig. 1(c)). This helps retain the context of the cellular structure during the visual analysis workflow. Throughout the course of this project, we iterated over different design choices for various elements of our visual analysis system. Following are some of the key design decisions that we had to make in the process.

• •

Initially, we used 400 colored circles along the circumference of the simulation domain to visualize the concentration values (Fig. 8(a)). However, as the overall system grew in size, we had to reduce the size of the individual views, which significantly shrunk the size of the 400 small circles. Increasing the size of the small circles led to overlapping among the spatially neighboring points (Fig. 8(b)). Therefore, we changed the visual design to use contiguous rectangular boxes instead of circles at each point (Fig. 8(c)). As a result, we can scale the boxes radially without worrying about overlapping (Fig. 8(d)).

• •

Deciding on an optimal design layout for the Parameter Control Board with 35 different parameter control bars was a challenging task. One straight-forward choice was to place the control bars for 35 parameters in a long vertical/horizontal panel with scrollbars to scroll through the list of individual control bars. However, the experts felt that it is important to have all the 35 bars visible at the same time, without the need to scroll around during the model calibration process.

• •

The next challenge was to find the best way of showing multiple recommended parameter values in the same view. Fig. 8(e) shows an instance in our current system with 3 recommended parameter values very close to each other. We lay them out vertically to make the close-by values standout. This helps the users in clicking on the texts and adjusting the parameter slider to that precise recommended value. Another choice to save space in the Parameter Control Board panel was to show the values only when the mouse hovers over the recommended nodes as shown in Fig. 8(f). However, we went with the first choice as it helps the experts to see all the recommended parameter values in the same view and not just the nodes in the bar. We believe that there could be better designs to address this, and we plan to explore other alternatives in the next version of our system.

Comparison with Previous Works: Previous visual analysis systems for simulation parameter exploration were all specifically designed to meet the requirements of their respective domain applications. However, one popular choice among some of the systems [44, 45, 5] was to project the high-dimensional space to a low-dimensional space for visualization. We did not opt for this choice because our experts were interested in directly manipulating in the exact parameter space rather than in some latent space. This helps them to explicitly map the effect of the parameters to different output regions. Using the reduced latent space can be confusing to interpret the meaning in the high-dimensional space. One unique feature of our proposed approach is that all the analysis tasks and navigation strategies [51] supported in our system are carried out using a single analysis framework in the backend, i.e, the trained neural network. Whereas, when using other surrogate models for prediction, most of these backend activities have to be carried out separately [7, 5, 8, 51]. In this work, we analyzed a simulation with 35 input parameters. To the best of our knowledge, previous visual parameter analysis systems [51, 44, 7, 60] did not have to deal with such large number of simulation parameters.

Implementation and Performance: We used the Keras python library (v2.2.4) [2], with TensorFlow backend, to implement and train our neural network-based surrogate model. We trained the model on a NVIDIA Pascal P100 GPU for 5000 epochs. It took 59.31 minutes to train the network for 5000 epochs and the final accuracy of the model was $87.6\%$. To perform post-hoc operations like sensitivity analysis and activation maximization on the trained network, we used the Keras-Vis [29], which is a high-level analysis library for neural networks. To perform uncertainty quantification for neural networks, we wrote custom code to turn on the dropout layers for trained Keras models during prediction/testing phase. Our frontend visual analysis system was designed using d3.js. We used flask framework [1] to interact with the trained neural network from our visual analysis system. The average time to get the predicted simulation output from the trained neural network for a new parameter configuration is 0.25 seconds, whereas, running the original simulation model for one configuration took 2.3 hours in a supercomputing cluster.

9 Conclusion and Future Work

In this paper, we have proposed an interactive visual analysis system to study and analyze a complex yeast cell polarization simulation model. The proposed system uses a trained neural network-based surrogate model as the backend analysis framework to facilitate interactive visual analysis. It allows the experts to interactively calibrate the simulation input parameters as well visually guide them towards discovering new parameter configurations. We also analyze the surrogate model to extract interesting insights about the original simulation model. We hope that our proposed approach can motivate researchers to look at neural networks as more than a prediction tool, and start utilizing them to conduct interesting analysis activities.

In future, we would like to extend the visual analysis framework to facilitate more complex analysis tasks like the recently proposed testing with concept activation vectors (TCAV) [27]. This will allow the experts to validate high-level domain specific concepts in the surrogate model. We plan to apply similar visual analytic approach for analyzing simulations from other application domains as well. As suggested by our experts, we also plan to simplify the model analysis and validation methods so that it is more intuitive for people without much machine learning background.

Acknowledgements.

This work was supported in part by US Department of Energy Los Alamos National Laboratory contract 47145 and UT-Battelle LLC contract 4000159447 program manager Laura Biven. We would also like to thank Marissa Renardy and Luke Andrejek for their help and feedback.