Stealing Neural Networks via Timing Side Channels

TL;DR

This paper demonstrates that timing side channel attacks can effectively extract neural network architectures and parameters, posing significant security risks for cloud-based AI services.

Contribution

It introduces a black box timing attack method using reinforcement learning and knowledge distillation to reconstruct neural networks, applicable across various architectures.

Findings

Successfully reconstructed models with high test accuracy

Attack scalable across different neural network architectures

Effective in inferring network depth via timing analysis

Abstract

Deep learning is gaining importance in many applications. However, Neural Networks face several security and privacy threats. This is particularly significant in the scenario where Cloud infrastructures deploy a service with Neural Network model at the back end. Here, an adversary can extract the Neural Network parameters, infer the regularization hyperparameter, identify if a data point was part of the training data, and generate effective transferable adversarial examples to evade classifiers. This paper shows how a Neural Network model is susceptible to timing side channel attack. In this paper, a black box Neural Network extraction attack is proposed by exploiting the timing side channels to infer the depth of the network. Although, constructing an equivalent architecture is a complex search problem, it is shown how Reinforcement Learning with knowledge distillation can effectively reduce the search space to infer a target model. The proposed approach has been tested with VGG architectures on CIFAR10 data set. It is observed that it is possible to reconstruct substitute models with test accuracy close to the target models and the proposed approach is scalable and independent of type of Neural Network architectures.

Tables

Table 1. TABLE I : Sample of Adversary’s Dataset. Depth of the architecture and the number of parameters as input features for the regressor.

Architecture	Parameters	Inference Time (s)
VGG16	138,357,544	0.59683408
	156,053,800	0.86338694
	133,048,360	0.49502961
VGG19	143,667,240	0.7642632
	168,441,896	1.11189311
	136,588,328	0.55131464

Table 2. TABLE II : Regression Model Predictions. The estimated depth of the neural network is rounded to nearest larger integer. The depth is correctly estimated using ensemble regressors: Random Forest (RF) and Boosted Decision Trees (BDT). The architecture of the model is in format: filters (kernel size) and MP is the Maxpool Layer

	Architecture	Parameters	Execution Time	Predicted Depth	True Depth
Model 1	[32(3), 32(3), MP, 64(3), 64(3), MP, 128(3), 128(3), MP]	309,290	0.036057	8.8 (RF); 8.1 (BDT)	9
Model 2	[32(3), 32(3), MP, 64(3), 64(3), MP, 128(3), 128(3), MP, 256(3), MP]	595,242	0.10738	10.15 (RF); 10.02 (BDT)	11
Model 3	[32(3), 32(3), MP, 64(3), 64(3), MP, 128(3), 128(3), 128(3), MP, 256(3), 256(3), MP]	1,334,442	0.18594	12.8 (RF); 12.6 (BDT)	13

Table 3. TABLE III : Evaluation of Reconstructed Models using Reinforcement Learning. The test accuracy of the reconstructed model is close to the original accuracy of the target model. The architecture of the model is in format: filters (kernel size) and GAP is the Global Average Pooling

	Reconstructed Architecture	Parameters	Original Accuracy	Reconstruction Accuracy
Model 1	[64(3), 32(5), 128(3), 64(5), 128(3), 64(5), 32(5), 128(3), GAP]	535,114	88.03%	86.06%
Model 2	[32(5), 32(5), 64(3), 32(3), 64(5), 128(3), 128(3), 32(3), 64(3), 64(5), GAP]	639,978	89.26%	85.65%
Model 3	[64(3), 128(3), 128(3), 32(5), 64(5), 128(3), 128(3), 32(3), 128(5), 128(3), 64(5), 128(5), GAP]	889,834	90.19%	85.3%

Table 4. TABLE IV : Evaluation of Regression Models using R 2 superscript 𝑅 2 R^{2} score and Mean Squared Error (MSE). Ensemble approaches estimate the depth better with higher R 2 superscript 𝑅 2 R^{2} score and lower MSE.

Regressor	Mean Squared Error	$R^2$ Score
Support Vector	7.5405	0.7295
Decision Tree	5.375	0.80719
Linear (Ridge)	4.46533	0.8398
Boosted Decision Tree	4.1947	0.8495
Random Forrest	3.7664	0.8648

Table 5. TABLE V : Same architecture trained on different datasets have different timing distribution.

Architecture	Dataset	Inference Time (s)
Alexnet	MNIST	0.28958
	CIFAR10	0.36527
VGG	MNIST	0.44178
	CIFAR10	0.63833

Equations

$$J({\theta }_c)=E_{P(a_{1:T};{\theta }_c)}[R]$$

$$\bigtriangledown_{\theta_{c}}J(\theta_{c})=\sum_{t=1}^{T}E_{P(a_{1:T};\theta_{c})}\big{[}\bigtriangledown_{\theta_{c}}\log P(a_{t}|a_{(t-1):1};\theta_{c})R\big{]}$$

$$L=\sum _{i=1}^n{\left(y_i^{target}-y_i^{template}\right)}^2$$

$$t_{res}=a\times t_{proc}+t_{net}+jitter$$

Full text

I Introduction

Of late, Neural Networks have been successfully employed to many diversified areas, namely computer vision, natural language processing and business intelligence [6]. Deep learning architectures have also been deployed for automating critical decision making in security applications like national critical infrastructures, malware and intrusion detection [42]. For various military applications such as unmanned combat aerial vehicle, automated target recognition and guided missile systems, the underlying decision making depends on state of the art deep learning architectures. Banks and financial services rely on deep learning techniques to process massive financial data. Autonomous driving has attracted several big automotive companies like Audi, Tesla and Waymo to invest billions of dollars into Deep Learning Research.

Designing and engineering Neural Networks for commercial services requires significant time, money and human effort, ranging from collection of massive data to fine-tuning the hyperparameters of the model for performance improvement. The commercial value of these models make them an important intellectual property for companies, due to which the model attributes such as number of layers, training algorithms and regularisation hyperparameter, are kept confidential as a black box. These black box models do not reveal any information to the service users other than the output predictions for the corresponding input. This has been commercialised as a business model by several cloud service providers such as Google, Amazon, Microsoft and BigML by deploying an end-to-end infrastructure for using Deep Neural Networks as a service. In Machine Learning as a Service (MLaaS) paradigm, trusted users submit training data to the service providers, who spend significant resources to design and train high performing models, deployed for public use on a pay-per-query basis. Despite its promise, the commercial value of the black box Neural Networks within MLaaS makes them susceptible to adversary’s attempts to extract the model functionality, model attributes, and circumventing the pay-per-query setting of the service. Given the architecture of Neural Networks, an adversary can further mount various privacy and security attacks like model inversion [9][14] and membership inference [32][31] to infer the input and training data instances and generate more accurate adversarial examples to evade classifiers during test phase [29]. These attacks violate the privacy of the sensitive data used for training the models and provide a way for the adversaries to evade security systems such as malware and intrusion detection systems thereby forcing to make incorrect predictions.

A major security question in such a black box setting such as MLaaS addressed in this paper is Can a weak adversary in a black box setting efficiently infer target Neural Network attributes by exploiting side channels with minimum number of queries? In this work, a novel model extraction attack in a black box setting is proposed by exploiting timing side channels and efficiently reconstructing a substitute model architecture with functionality close to the target model using a constant number of queries.

Key Challenges in Model Extraction Attacks. Stealing a Neural Network architecture and its functionality is a challenging problem owing to the large number of hyperparameters making brute-force infeasible. The rapid growth of Neural Network design space has increased the complexity of architectures making the problem of black-box model extraction more challenging. In the black box setting as in MLaaS, the adversary has only access to the output predictions given input and lacks any knowledge about the model and the training data. Previously, model extraction attacks have relied on using input-output relations to identify the decision boundary of the target model [38][27]. However, such attacks require significant computational resources and incur a huge time overhead to search for the substitute model. For instance, given a prior knowledge about the number of layers and type of layers in a Neural Network, it still takes 40 GPU days to search for a simple 7 layer networks architecture [27]. Further, these attacks do not accommodate the state of the art architectures with complex topologies and skip connections [18]. While extracting the model, these attacks require large number of queries which grow with the size of the architecture making the attacks highly inefficient [25][38]. An alternative approach for model extraction is to exploit side channels like power consumption [3], memory access patterns [17][16] and cache side channel attacks [43][15] to infer target model attributes. While these attacks give fine grained information about the target model during execution, the threat model requires escalated adversary privileges and strong assumptions like physical access to hardware and shared resources for processes running on the server.

Proposed Approach. The objective of model extraction attack is to search for a substitute model with similar functionality as the target neural architecture. However, the search space for the substitute model is very large and complex due to the large number of hyperparameters in Neural Networks. To make the search tractable and efficient, the adversary has to reduce the search space by identifying some of the attributes of the target Neural Network in a black box setting using minimum queries. In a black box setting, a weak adversary can obtain the output prediction corresponding to a given input image. This paper shows the existence of timing side channels in the black box setting due to the dependence of the total execution time of the Neural Network on the total number of layers or depth of the network. From the total execution time, an adversary can infer the total number of layers (depth) of the Neural Network using a regressor trained on the data containing the variation of execution time with Neural Network depth. This additional side channel information obtained, namely the depth of the network, reduces the search space for finding the substitute model with functionality close to the target model.

To efficiently search for the optimal Neural Network, an optimisation problem is introduced which is solved using Reinforcement Learning based Neural Architecture Search. A Recurrent Neural Network(RNN) controller predicts a new substitute architecture, whose performance determines the reward to improve the controller’s prediction for subsequent optimisation epochs [44]. The optimisation problem involves minimising the loss function computed between the predicted labels of the substitute model and the target model instead of the true labels [13]. This ensures that the substitute model learns to mimic the predictions of the target model and hence, increasing the similarity of the two models. Over multiple epochs of training, the RNN controller updates its parameters based on the reward obtained by minimising the loss, to predict optimal substitute architectures with performance close to target model. The proposed architecture search and reconstruction technique can be used with other attack approaches as well, like cache side channel attacks [15][43].

The proposed approach assumes a weak adversary with only black box access to the target Neural Network and requires a constant number of queries to infer the Neural Network depth independently of the architecture size. Further, the objective function of Reinforcement Learning maximises the test accuracy of the proposed Neural Network which ensures that the final substitute Neural Network is optimal and close to the target model.

Evaluation. To measure the success of the proposed model extraction attack, the performance of the regressor to correctly infer the depth of the Neural Network given the total execution time is shown. The performance of different regressors, to infer the Neural Network depth has been assessed based on the $R^{2}$ score and the Mean Squared Error to select the regressor model which captures the maximum variance and accuracy. From the results, ensemble based regressors like random forrest and boosted decision trees outperform their linear counterparts: Ridge regression, Support Vector Machine (SVM) and decision trees. This is followed by the evaluation of the Reinforcement Learning based architecture search by comparing the test accuracy of the reconstructed model with the target model accuracy. The experiments are performed using deep convolutional Neural Networks similar to VGG architectures [34]. The proposed Reinforcement Learning based architecture search technique can generate a model with test accuracy within 5% of the target model.

Main Contributions. The paper makes the following main contributions:

• •

Shows that Neural Networks are vulnerable to timing side channel attacks as Neural Network architectures with different depth have different execution time (Section V).

• •

Proposes a novel attack to infer the depth of the Neural Network using timing side channels in constant number of queries in a black box setting (Section VI).

• •

Proposes an efficient search technique to reconstruct an optimal substitute architecture using Reinforcement Learning while ensuring functionality similar to the target model (Section VII).

II Background

II-A Neural Networks

Let $(x,y)$ be the data points obtained by sampling from a probability distribution $D$ over the space $X$ of input feature values and space $Y$ of output labels. The goal of machine learning algorithms is to learn the mapping from $X$ to $Y$ captured by a function $f:X\rightarrow Y$. The associated loss function $l_{f}:X\times Y\rightarrow\mathbb{R}$ captures the error made by the prediction $f(x)$ when the true label is $y$. Deep Learning, a subset of machine learning algorithms, uses Deep Neural networks modelled as function $f_{h}(x,\theta)$ where $\theta$ are the parameters optimised during training to obtain minimum expected loss under the constraint of the hyperparameters $h$. The set of hyperparameters $h$ includes the depth of the Neural Network, stride and filter size of convolution and maxpool layers and regularization hyperparameters. The performance of the Neural Networks is measured by computing the accuracy on test data in classification tasks.

II-B Security and Privacy in Machine Learning

Machine learning is known to have several security and privacy issues in adversarial settings [8]. A major security threat in Neural Networks is Adversarial examples, i.e, perturbed data instances that fool the classifier into misclassifying the image by either poisoning the training data or evading the decision logic during inference [29]. For a Neural Network function $f_{h}(x;\theta)$ with input data point $x$ and parameters $\theta$, an adversary can violate the confidentiality and privacy of input data ($x$), training data, model parameters ($\theta$) and the model computation ($f_{h}(x;\theta)$). The privacy of input passed to the model can be violated using model inversion attack [9] by extracting the input when the adversary knows the output, parameters and gradients. Another major class of privacy attacks is Membership inference which violate the privacy of individual members of the training dataset by identifying whether a given data point is in the dataset or not [32][31]. Further, the computation of the Neural Networks leak information in the form of side channels which allow the adversary to extract model details or inputs [3][40]. Machine learning models can be extracted by adversary to reconstruct a substitute model with similar functionality as the target model and hence violating the intellectual property of the service provider [38][40][39]. However, some of these attacks assume that the underlying Neural Network architecture is known to the adversary. Hence, extracting the target model architecture enables the adversary to mount further security and privacy attacks.

Implementation and physical characteristics of systems expose information about the underlying computation which can be extracted in the form of side channels. Power consumption and Timing side channels are some common manifestations of side channel attacks overlooked during system design. While side channels like power channels are accurate and reveal significant information about the target architecture, they require expensive equipment and probes to monitor and measure the power [22]. Timing Channels arise when the program uses conditional statements dependent on the secret information which influences the runtime or when the access timing correlates strongly with the program locality dependent on the secret [21].

III Problem Statement

Model Extraction. Given a black box access to a target Neural Network $f_{target}$, the goal of the adversary is to search for a substitute Neural Network $f_{substitute}\in S$, where $S$ is the search space for all possible Neural Network models with different hyperparameters, such that the functionality of $f_{target}$ approximates $f_{substitute}$ using minimum possible queries. The metric used to measure the functionality of the models, $f_{substitute}$ and $f_{target}$, is the test accuracy ($R_{test}$). In other words, the objective is to minimize the difference in test accuracy $R_{test}=\frac{1}{|D|}\sum_{(x,y)\in D}d(f_{target}(x),f_{substitute}(x))$ between the two models $f_{target}$ and $f_{substitute}$ for inputs ($x,y$) sampled from the data ($D$), i.e, $(x,y)\stackrel{{\scriptstyle i.i.d}}{{\sim}}D$ and a given distance function $d$ between the two inputs.

Exploiting Timing Side Channels. To reduce the entropy of search space of possible Neural Network models and make the search more efficient, the adversary exploits the timing side channels to infer the number of layers from the total execution time. For this, the adversary collects a dataset ($D_{A}$) with execution time ($T$) and Neural Network depth ($K$) for various models by varying the number of parameters. Formally, given the attacker dataset $D_{A}$ = {$(T_{1},K_{1}),\cdots,(T_{N},K_{N})\}$ of i.i.d. random variables, for a given depth of the target Neural Network ($k$) from the total execution time ($t$), the adversary estimates the regression function $R(k)=E\{K|T=t\}$.

Model Search. The estimated depth is used to constraint the search space to $S_{k}$ which is the set of all the Neural Network models of depth $k$. This allows the adversary to search for the substitute model $f_{substitute}$ in the search space $S_{k}$ instead of search space $S$ where $S_{k}\subset S$. However, the search space $S_{k}$ is parameterised by kernel size, stride and number of filters which still make the search space large. To make the search space tractable, the adversary uses Reinforcement Learning paradigm where the accuracy of the target model is included in the objective function as part of the reward, to search for Neural Networks with higher test accuracy.

IV Threat Model

Setting. There are two settings for machine learning in adversarial setting: white box and black box, based on the adversary’s knowledge about the target system. In white box setting, the adversary has access to the underlying data, learning algorithms, architecture of the model, training parameters and target model architectures which allows the adversary to compute the output of the intermediate layers. The proposed attack is in a black box setting where the adversary is weak and does not have access to the model internals and can only query the trained model and obtain the corresponding output predictions.

Hardware. The proposed attack is an inference phase attack, where the trained target model has been deployed as a service to the users. During inference, CPUs and Neural Network accelerators are extensively used while GPUs are used for training Neural Network architectures [37]. Majority of machine learning Cloud service providers use CPUs [12]. The attack is evaluated using CPUs but the approach can be extended to other hardware accelerators as well. The adversary requires the same processor as the target model which can be openly obtained in most of the ML as a Service (MLaaS) platforms like Amazon Sagemaker and Facebook which heavily rely on CPUs for the inference and provide the hardware specifications [1][2]. The target hardware or the service can be purchased by adversary to run the queries and generate the attack dataset which is a one time operation and done as part of the setup phase for the attack.

Data. The attack assumes a weak adversary with no knowledge about the training data and only knows the input-output dimensions and range of values they can take. The attacker, however, is assumed to know the underlying data distribution from which the training data was sampled. This allows the attacker to sample data points as inputs and pass them to the target model for predictions. There are two main approaches to reconstruct the training data for the substitute model: Iterative membership inference attacks and data reconstruction attacks. In case of membership inference attack, the adversary samples data points from the underlying data distribution $x\sim D$ which is passed as a query to the target model from which the adversary obtains the model output posterior $f(x)$. Given the output posterior of the model for the input, the attacker checks if the value of the maximum posterior is greater than a threshold($x$ $\in$ $D_{T}$) or not($x$ $\not\in$ $D_{T}$) [31]. This is done iteratively by sampling data points and using membership inference attacks to reconstruct the training data used by the target Neural Network. In data reconstruction attack, an adversary uses a generative adversarial network to reconstruct training data samples from target model by finding the approximate training data distribution [30][10]. Either of the two approaches can be used to reconstruct the dataset and it is a one time operation done during the setup phase of the attack as described in Section VI.

V Timing Channels In Neural Networks

There exists a direct relation between execution time of Neural Networks and their dependence on various hyperparameters for different Neural Network layers as shown in this section. This dependence of the execution time on the Neural Network hyperparameters allows an adversary to infer the architecture details using the total execution time which forms the basis of the attack. A typical Convolutional Neural Network has three types of layers based on operations: convolution layer, maxpool layer and fully connected layer and each of the layer has stride, kernel size and number of filter as the hyperparameters.

Convolution Layer. Convolution is a weighted sum operation which computes the multiplication of the parameters ($\theta_{ij}^{(l)}$) of layer $l$ and the input feature map ($z_{ij}^{l-1}$) and adds the results, $x_{i}^{(l)}=\sum_{i}\sum_{j}\theta_{ij}^{(l)}\times z_{ij}^{(l-1)}$. The execution time of convolution layers is proportional to the number of multiplications (shown in Figure 2(a)) which is given by: $o_{w}\times o_{h}\times c_{o}\times f_{w}\times f_{h}\times c_{i}$ where $o_{w}$ and $o_{h}$ are the output matrix width and height, $c_{i}$ and $c_{o}$ are the input and output number of channels and $f_{h}$ and $f_{w}$ represent the filter width and filter height.

Maxpool Layer. Maxpool computes the maximum of the $k\times k$ region of preceding layer feature map where $k$ is the size of the kernel. Hence, the computation of maxpool depends on the number of filters of the feature map and the kernel size ($k$). The execution time increases with increasing filter size and increasing kernel size as shown in Figure 2 (b). For stride, there is an inverse relation between the execution time and the stride as shown in Figure 2 (c). The output size decreases with increasing stride which results in a decrease in the execution time due to fewer number of multiplications.

Fully Connected Layer. Fully connected layer performs a matrix vector multiplication between the parameters of the Neural Network and the input image map. Formally, the matrix vector multiplication of parameters $\theta_{ij}^{(l)}$ for layer $l$ with the activation of previous layer $z_{j}^{(l-1)}$ is given by $x_{i}^{(l)}=\sum_{j}\theta_{ij}^{(l)}\times z_{j}^{(l-1)}$. Given two fully connected layers of input nodes $m$ and output nodes $n$, the execution time varies linearly with the total number of multiplications, $m\times n$ and the linear relation can be seen in Figure 3 (a).

Variation with Depth. Neural Networks are embarrassingly parallel and all the computations in one layer can be executed in parallel to some extent which enables optimisations like model and data parallelism to improve the performance [24]. However, due to sequential computation of Neural Networks, the total execution time is the sum of the execution time of individual layers. In Figure 3 (b) and Figure 3 (c), the execution time increases linearly with the increase in the network architecture depth for both simple and complex topologies. The simple topologies of Neural Networks include LeNet, AlexNet and VGG architectures while the complex topologies include Resnet architectures which have skip connections between the layers. This linear relation between the number of layers and the total execution time forms the basis of the attack methodology to infer the depth of the Neural Network given the execution time.

Hardware Factors. While the Neural Networks fetch the data and parameters from the memory, the program could incur latencies during reading or storing data, contention of threads and inefficient caching. Since the hardware used for all the models is same, these factors are assumed to effect the execution time of all the Neural Networks in the same manner.

VI Attack Methodology

Side channels reveal only a part of the secret in the target system and identifying the rest of the secret is modelled as a search problem [43]. The proposed attack is broadly divided into three phases as shown in Figure 1:

• •

Setup Phase: Adversary aggregates the dataset by measuring the execution time of multiple models with different hyperparameters, on a particular hardware, to be used in the actual attack. Further, the adversary reconstructs the training dataset using iterative membership inference [31] or dataset reconstruction attack [30]. This is a one time operation required to be performed prior to the attack.

• •

Attack Phase: Adversary queries the target Neural Network and measures the total execution time averaged over all queries. A regressor is trained on the attacker dataset which is used to infer the target Neural Network depth.

• •

Reconstruction Phase: Adversary searches for an optimal Neural Network with test accuracy close to that of the target Neural Network model within the reduced search space by making the depth of the Neural Network constant as inferred from the attack.

VI-A Setup Phase

The adversary during the setup phase reconstructs the training data with the prediction of target model ($f_{target_{i}}(x)$) as labels instead of true labels for model distillation and creates a dataset containing the execution time of Neural Network architectures with varying hyperparameters. This a one time operation required prior to performing the attack.

Reconstructing Training Data. Since, the attack assumes a weak adversary with no knowledge about the training data, the adversary needs to identify and generate data samples used as the training dataset. The underlying data distribution is assumed to be known, the adversary can sample data points $x$ from the underlying data distribution. The adversary queries the target model with data sample $x$, and based on the target model output posterior, determines if the data sample belongs to the training data $D_{T}$ of the target Neural Network or not, by selecting a suitable threshold [31]. Given the knowledge of the data samples used as part of the target model training dataset, the adversary labels the input instances ($x$) with the predictions of the target model ($f_{target}(x)$) instead of the true label ($y$). The aggregated training data $(x_{i},f_{target}(x_{i}))$ includes soft target model predictions ($f_{target}(x_{i})$) instead of hard labels ($y$), which ensures that the substitute model learns to mimic the functionality of the target model [13][5].

Creating Attack Model Timing Dataset. The adversary populates the attack model dataset which contains the time taken for inference and corresponding depth of the Neural Networks along with the number of parameters for the corresponding model as shown in Table I. Formally, for different model architectures and depth ($K$), the attacker collects the corresponding execution time ($T$) to generate the attacker dataset $D_{A}$ = {$(T_{1},K_{1}),\cdots,(T_{N},K_{N})\}$. This dataset is specific to a particular hardware and the collected data can then be used for stealing any model run on the same hardware.

VI-B Attack Phase

Query. The adversary sends queries to the target model and computes the overall execution time averaged across all the queries. For each additional query of model extraction attacks proposed previously, a new bit of information is leaked which reduces the entropy of the target black box model. However, this requires a large number of queries due to the large number of parameters of deep Neural Network architectures. In the proposed attack, each of the query to the target model reveals the same bit of information (execution time averaged across all queries) which allows to make constant number of queries, independent of the architecture.

Regression. The average execution time measured from the target Neural Network ($t$) is used to estimate the depth of Neural Network ($k$) using regressor trained on the attack model dataset created during the setup phase, $R(k)=E\{K|T=t\}$. For current experiments, the attacker dataset for training the regressor model is created using 100 Neural Networks with different depth and parameters as shown in Figure 4.

VI-C Reconstruction Phase

The search space of all possible Neural Networks is very large and complex due to which designing the Neural Network manually is hard and requires optimal search strategies to reduce and simplify the search space. The depth of the Neural Network reduces the search space and the model exploration is automated using Reinforcement Learning which outputs the best model architecture in the constrained search space [44]. A Recurrent Neural Network (RNN) based controller predicts the hyperparameters of each layer in the template Neural Network sampled using the reward from the previous proposed architecture. The parameters of controller RNN $\theta_{c}$ are then optimised based on performance of the predicted architecture using policy gradient method.

Formally, the controller predicts the architectures through actions $a_{1:T}$ and the predicted model tries to achieve accuracy $R$ which is used to compute the reward signal to train the controller. The policy function maps the proposed architecture to a real number (accuracy) which is used to compute the reward for the RNN controller. The policy weights are optimised to ensure that newer architectures proposed by the RNN controller have a higher performance. After training the RNN controller for multiple iterations, the proposed substitute architecture is optimal, i.e, has the highest expected accuracy among all the models within the search space. The goal is to maximise the expected accuracy of sampled architecture given by,

[TABLE]

while minimising the difference in test accuracy between the target and proposed substitute model, $R^{target}_{test}-R^{substitute}_{test}$.

Since the reward signal $R$ is non-differentiable, we need to use a policy gradient method to iteratively update the parameters of the RNN controller $\theta_{c}$. This work uses the $REINFORCE$ algorithm to directly update the policy weights using stochastic gradient ascent and improve the policy [41]:

[TABLE]

The training of each proposed model is done using knowledge distillation where the loss function for training the substitute model is the distillation L2 loss function [13] between the substitute model predictions ($y^{template}$) and target model($y^{target}$) predictions instead of the true labels($y$) for a given data point $x$ $\epsilon$ $D$ and is given as,

[TABLE]

For each iteration of training the substitute model, the substitute model learns to mimic the predictions of the target model making the test performance similar (Figure 5).

VII Evaluation

Data. For all the experiments, CIFAR10 dataset [23] is used which contains 60,000 32x32 colour images in 10 classes with around 6000 images per class and the classes are mutually exclusive. For training, 50,000 images are used while 10,000 images are used for testing.

Experiment Setup. The processor used for evaluation and experimentation is Intel Xeon Gold 5115 server processor with a 2.4GHz clock speed, 196GB of main memory and 40 cores. All the reported number are an average of 20 inference runs. Accurate timing of the Neural Network inference is done using $process\_time()$ from the $time()$ python library which computes the time interval for running the inference using the CPU counter and is not effected by execution of other unrelated processes. The clock has a tick rate (ticks/s) of 10,000,000 which indicates a high resolution of 1e-07.

VII-A Regression

Ideally, on executing a Neural Network on a hardware accelerator, the total execution time depends solely on the number of layers due to their sequential computation. Hence, varying the number of neurons or filters in a particular layer in a deep Neural Network should not change the overall execution time as individual computations within the same layer can be performed in parallel. This property of Neural Networks makes them embarrassingly parallel [37]. However, in practice, a variation in number of parameters within a particular layer shows a deviation in execution time due to inefficient parallelism as seen in Figure 4.

To address this, it is important to train a good regressor which captures the maximum variance in the timing dataset. A good regressor should be able to capture and explain the variance of the dataset through its predictions and generalize well over the data samples. The evaluation of five different regressor models on the attacker dataset is done using $R^{2}$ score metric to measure the variance explained by the model and the mean squared error (MSE) to measure the error in estimating the depth of the network.

Ridge and Support Vector regressors are linear models, while Decision Tree Regressor builds a tree structure over different features in the dataset by minimising the standard deviation. Boosting is an ensemble technique to combine multiple sequential decision trees and fit the data, while improving the error from the pervious model. On the other hand, Random Forest regressor fits subsets of data over multiple decision trees independently and outputs the mean of prediction from each of the model.

Based on the results, ensemble approaches like boosted decision trees (BDT) and random forrest (RF) regressor have a higher $R^{2}$ score and lower mean squared error to estimate the depth more accurately as compared to the linear models which fail to capture the variance of the attacker dataset (Table IV). The estimated depth of the ensemble regressors on the target deep Neural Networks from the corresponding execution time on VGG based target Neural Network architectures is shown in Table II. Since the output of the regressor is a continuous variable, the prediction of the regressor is rounded to the nearest larger integer. For all the three Neural Networks used for evaluation, the regressors estimate the correct depth from the total execution time.

VII-B Reconstruction using Reinforcement Learning

Once the adversary has estimated the depth of the Neural Network, the information is used to reduce the search space. Now, the adversary has to search for the optimal substitute Neural Network with test accuracy close to the target Neural Network.

The Reinforcement Learning based architecture search is evaluated by fixing the depth of the model architectures inferred using the regressor. The search is further constrained by specifying the convolutional layer parameter range for the kernel size ($k$) $\epsilon$ {3,5} and the number of filter ($n_{f}$) $\epsilon$ {32,64,128} which are commonly used hyperparameters values used in all state of the art networks [34]. The architecture search approach explores the space of 50 models, and outputs the architecture corresponding the highest accuracy. To improve the performance of the substitute model, we use fully convolutional net architecture by replacing maxpool layer with convolutional layers with higher stride [35]. The reward used for updating the controller is the maximum validation accuracy of the last 5 epochs cubed which is clipped in the range (-0.05, 0.05) to ensure that the gradients do not overshoot. The controller RNN includes 1 LSTM cell and 32 hidden units and each proposed template model is trained for 20 epochs. For all the three model, the test accuracy of the substitute model generated is within 5% of the target model architecture as shown in Table III.

VIII Mitigation

The main reason for the manifestation of timing side channels in Neural Networks is the sequential computation of layers which determine the total execution time. It is important to design Neural Networks resistant to timing side channels to prevent model extraction. Some of the possible defences are discussed in this section.

Adding Noise to Execution Time. Instead of having a Neural Network with timing dependent on the depth of the Neural Network, one defence mechanism is to design Neural Networks without the dependency of the execution time on the number of layers and hyperparameters. Additional noise to the total execution time can be added in the form of latency by including dummy computations and layers. However, this results in a model security and utility tradeoff which is a concern in real time critical applications where the performance of the model within time constraints is vital.

Adversarial Machine Learning. The second mechanism is a training phase defence where the attack and defence game can be viewed as an adversarial machine learning problem. The goal of the attacker is to fit the best possible curve or function to the attacker dataset for regression while the goal of the defender is to poison the dataset with wrong data instances such that the regressor makes wrong predictions. The defender injects adversarial examples with incorrect timing and depth values resulting in incorrect regressor estimation.

IX Discussion

Variation Across Datasets. The timing distribution for Neural Networks is specific to the training data used. For the same architecture, using different datasets results in different execution time due to different number of computations as shown in Table V.

Different dataset have images of different sizes and properties due to which the intermediate input feature maps require different number of multiplications. For instance, MNIST dataset has images of size 28 $\times$ 28 $\times$ 1, while CIFAR10 dataset has images of size 32 $\times$ 32 $\times$ 3 which results in difference in number of computations to be performed. Hence, the regressor is specific to a particular timing distribution unique to a dataset and a different attack model has to be trained to fit different timing distribution.

Extending to Remote Setting. While the evaluation the attack is on a local model, this can be extended to remote setting like in MLaaS where the model is deployed on a Cloud server. The total round trip time in case of remote setting also includes some additional noise during propagation in the form of jitter as well as the time taken for network propagation [7]. A round trip time model for remote timing attacks is given below where the total response time ($t_{res}$) is a linear function of the scaled processing time ($t_{proc}$), network propagation time ($t_{net}$) and the jitter.

[TABLE]

To extract the processing time of the Neural Network from the total round trip time, the additional network time and jitter have to be estimated and filter them from the round trip time.

Model Extraction Defences. Several defences have been proposed to mitigate the attacks that exploit the information from output predictions. Suppressing the information provided by output logits reduces the accuracy of the substitute model but degrades the utility [38]. Stateful defence mechanism to monitor and detect a variation in the input query distribution [19] or raise an alert if the information gained by an adversary exceeds a threshold [20]. Trusted hardware like Intel SGX can protect the confidentiality and integrity of the model by moving the model offline to the user’s system [11]. For attacks that rely on memory access patterns, implementation using Oblivious RAM could help to hide the access pattern [36]. All the defences mentioned are proposed for attacks that use the output prediction scores but none of these approaches can help to mitigate timing side channels.

X Related Work

Deep Neural Network attributes can be extracted from the input-output relationship of the target network [33][28]. For example, given a black box neural network, an adversary queries the model with data instances $x$ to obtain corresponding output predictions $f(x)$. These input-output pair ($x$,$f(x)$) is used for training the substitute model, whose parameters and hyperparameters can be computed by solving a system of linear equations between the input, output predictions and the unknown parameters [38]. Further, other hyperparameters of the objective function can be solved by finding an approximate solution to a system of overdetermined equations [39]. Unfortunately, such attacks rely on large number queries since each new query provides the solution for a different unknown variable. Further, for very deep neural network architectures with large number of layers and millions of parameters, the computation cost is high. Unlike these attacks, timing side channels can be exploited in constant number of queries since each query reveals the same model attribute and the approach is scalable to deep networks with millions of parameters.

Several machine learning models trained to predict the model attributes based on the input output relationships can infer significant number of model attributes [27]. However, such techniques incur a high computational cost for training large number of machine learning attack models. For a simple digit classification task, it takes 10k attack models trained over 40 GPU days.

Alternatively, Side Channel leakage during the model execution provides fine-grained information about the target model in the form of cache misses, memory access pattern [26][16], power consumption profile [3] and hardware performance counters [4]. The read-after-write dependencies for the inputs activation filters and output activation filters are different which reveal the dimensions and type of individual layers [17]. Assuming shared resources between the target model (victim)process and the attacker process, an adversary can monitor the number of calls, the size of matrix dimensions to identify the number of layers and hyperparameter details in the Neural Network [15]. Further, cache attacks can distinguish different activation function like relu, sigmoid and tanh by monitoring the probe addresses [43]. Given the power traces during the execution of models, algorithms like differential power analysis, correlated power analysis and horizontal power analysis can be used to extract the the number of parameters in each layers, values of each parameters, total number of layers and the type of activation function [3].

However, these side channel attacks either assume a strong adversary with physical access to the hardware or require shared resources between the processes. Unlike these side channel attacks, the proposed approach considers a weak adversary with (remote) blackbox access to the target model.

XI Conclusions

This paper shows that Neural Networks are vulnerable to timing side channels attacks as the total execution time depends on the sequential computation along the number of layers or depth. For a weak adversary in a black box setting, the timing channel vulnerability can be exploited to infer the depth of the Neural Network architecture. The evaluation of various regressors on the timing data shows that the ensemble based regressors perform better than their linear counterparts based on the $R^{2}$ score and Mean Score Error values. Further, the search problem of extracting a Neural Network architecture by exploiting side channels can be addressed efficiently using Reinforcement Learning. This approach can be used with other attacks like cache attacks and memory access pattern monitoring to accurately identify the substitute model close to the target model. The attack is evaluated on VGG like deep learning architectures and it is shown that a substitute model can be reconstructed within 5% of the test accuracy of the target Neural Network.