A Flexible Privacy-preserving Framework for Singular Value Decomposition under Internet of Things Environment

TL;DR

This paper introduces a flexible, privacy-preserving fog computing framework for singular value decomposition (SVD) tailored for IoT environments, enabling secure data analysis at the network edge.

Contribution

It proposes a novel framework that ensures privacy in SVD computations within IoT fog environments, adaptable to various application needs.

Findings

Framework demonstrates practical security and performance.

Supports multiple application objectives with flexible design.

Effective in real IoT scenarios.

Abstract

The singular value decomposition (SVD) is a widely used matrix factorization tool which underlies plenty of useful applications, e.g. recommendation system, abnormal detection and data compression. Under the environment of emerging Internet of Things (IoT), there would be an increasing demand for data analysis to better human's lives and create new economic growth points. Moreover, due to the large scope of IoT, most of the data analysis work should be done in the network edge, i.e. handled by fog computing. However, the devices which provide fog computing may not be trustable while the data privacy is often the significant concern of the IoT application users. Thus, when performing SVD for data analysis purpose, the privacy of user data should be preserved. Based on the above reasons, in this paper, we propose a privacy-preserving fog computing framework for SVD computation. The security and performance analysis shows the practicability of the proposed framework. Furthermore, since different applications may utilize the result of SVD operation in different ways, three applications with different objectives are introduced to show how the framework could flexibly achieve the purposes of different applications, which indicates the flexibility of the design.

Tables

Table 1. Table 1: Computational Cost of the Proposed Framework

Entity	Computational Cost (milliseconds)
ED	$2\times 55.493\cdot N_C$
FDs	$55.493\cdot N\cdot N_C$
SDd	$55.493\cdot N\cdot N_C$

Table 2. Table 2: Communication Overhead of the Proposed Framework

Communication Flow	Bit Length of Message
ED$\to$ FD	$N_C\cdot 2\left|n\right|$
FDs $\to$ SDd	$N\cdot N_C\cdot 2\left|n\right|$
SDd $\to$ SDu	$l^2\left(2{\kappa }_1+2{\kappa }_3+{\kappa }_N\right)$
SDd $\to$ SDv	$N^2\left(2{\kappa }_1+2{\kappa }_3+{\kappa }_l\right)$

Equations

$$\begin{array}{r}\mathbb{A}\cdot {\mathbb{A}}^T=\mathbb{U}\Sigma {\mathbb{V}}^T\mathbb{V}{\Sigma }^T{\mathbb{U}}^T=\mathbb{U}\Sigma {\Sigma }^T{\mathbb{U}}^T\\ {\mathbb{A}}^T\cdot \mathbb{A}=\mathbb{V}{\Sigma }^T{\mathbb{U}}^T\mathbb{U}\Sigma {\mathbb{V}}^T=\mathbb{V}{\Sigma }^T\Sigma {\mathbb{V}}^T\end{array}$$

$$\mathbb{A}=\left(\begin{array}{cccc}{d}_{11}& d_{12}& \dots & d_{1N}\\ d_{21}& d_{22}& \dots & d_{2N}\\ ⋮& ⋮& \ddots & ⋮\\ d_{l1}& d_{l2}& \dots & d_{lN}\end{array}\right)$$

$$m_i=a_1{d}_{1i}+a_2{d}_{2i}+\cdots +a_l{d}_{li}$$

$$C_i=g^{m_i}\cdot r_i^n\mathrm{mod}{n}^2$$

$$\begin{array}{rl}{C_i}^{{}^{\prime }}& =C_i\cdot g^{rz}\mathrm{mod}{n}^2\\ & =g^{{\sum }_{k=1}^l{a}_k\cdot d_{ki}}\cdot g^{{\sum }_{k=1}^l{a}_k\cdot (z_{ki}\cdot W+r_{ki}\cdot S)}\cdot r_i^n\mathrm{mod}{n}^2\\ & =g^{{\sum }_{k=1}^l{a}_k\cdot (d_{ki}+z_{ki}\cdot W+r_{ki}\cdot S)}\cdot r_i^n\mathrm{mod}{n}^2\end{array}$$

$${m_i}^{{}^{\prime }}=\sum _{k=1}^l{a}_k\cdot (d_{ki}+z_{ki}\cdot W+r_{ki}\cdot S)\mathrm{mod}n$$

$${\mathbb{A}}^{{}^{\prime }}=\left(\begin{array}{cccc}{d}_{11}^{{}^{\prime }}& d_{12}^{{}^{\prime }}& \dots & d_{1N}^{{}^{\prime }}\\ d_{21}^{{}^{\prime }}& d_{22}^{{}^{\prime }}& \dots & d_{2N}^{{}^{\prime }}\\ ⋮& ⋮& \ddots & ⋮\\ d_{l1}^{{}^{\prime }}& d_{l2}^{{}^{\prime }}& \dots & d_{lN}^{{}^{\prime }}\end{array}\right)$$

$$\begin{array}{rl}& a_1\cdot (d_{1i}+z_{1i}\cdot W+r_{1i}\cdot S)+\cdots +a_{l-1}\cdot (d_{(l-1)i}+z_{(l-1)i}\cdot W+r_{(l-1)i}\cdot S)\\ <& a_1\cdot (d+tW+tS)+\cdots +a_{l-1}\cdot (d+tW+tS)\\ =& \sum _{k=1}^{l-1}{a}_k\cdot (d+tW+tS)<a_l\end{array}$$

$$\begin{array}{rl}\frac{X_l-X_{l-1}}{a_l}& =\frac{a_l\cdot (d_{li}+z_{li}\cdot W+r_{li}\cdot S)}{a_l}\\ & =d_{li}+z_{li}\cdot W+r_{li}\cdot S\end{array}$$

$$e_u={e_u}^{{}^{\prime }}\mathrm{mod}S\mathrm{mod}W$$

$$\begin{array}{rl}{e_u}_{ij}^{{}^{\prime }}=\sum _{k=1}^N{d}_{ik}^{{}^{\prime }}\cdot d_{jk}^{{}^{\prime }}& =\sum _{k=1}^N(d_{ik}+z_{ik}\cdot W+r_{ik}\cdot S)\cdot (d_{jk}+z_{jk}\cdot W+r_{jk}\cdot S)\\ & =\sum _{k=1}^N{d}_{ik}\cdot d_{jk}+\sum _{k=1}^N[(z_{ik}{d}_{jk}+z_{jk}{d}_{ik})W+z_{ik}{z}_{jk}{W}^2]\\ & +S\sum _{k=1}^N[(r_{ik}{d}_{jk}+r_{jk}{d}_{ik})+(z_{ik}{r}_{jk}+z_{jk}{r}_{ik})W+r_{ik}{r}_{jk}S]\end{array}$$

$$\sum _{k=1}^N{d}_{ik}{d}_{jk}+\sum _{k=1}^N[(z_{ik}{d}_{jk}+z_{jk}{d}_{ik})W+z_{ik}{z}_{jk}{W}^2]<N(d^2+2tdW+t^2{W}^2)<S,$$

$$d_{ij}={d_{ij}}^{{}^{\prime }}\mathrm{mod}S\mathrm{mod}W$$

$$\mathbb{U}=\left(\begin{array}{cccc}{u}_{11}& u_{12}& \dots & u_{1l}\\ u_{21}& u_{22}& \dots & u_{2l}\\ ⋮& ⋮& \ddots & ⋮\\ u_{l1}& u_{l2}& \dots & u_{ll}\end{array}\right)$$

$$\Sigma =\left(\begin{array}{cccccc}{\sigma }_1& 0& \dots & 0& 0& 0\\ 0& {\sigma }_2& \dots & 0& 0& 0\\ ⋮& ⋮& \ddots & ⋮& ⋮& ⋮\\ 0& 0& \dots & {\sigma }_{\lambda }& 0& 0\\ ⋮& ⋮& & ⋮& ⋮& ⋮\\ 0& 0& \dots & 0& 0& 0\end{array}\right)$$

$$\mathbb{U}\Sigma =\left(\begin{array}{cccccc}{\sigma }_1{u}_{11}& {\sigma }_2{u}_{12}& \dots & {\sigma }_{\lambda }{u}_{1\lambda }& 0& 0\\ {\sigma }_1{u}_{21}& {\sigma }_2{u}_{22}& \dots & {\sigma }_{\lambda }{u}_{2\lambda }& 0& 0\\ ⋮& ⋮& \ddots & ⋮& ⋮& ⋮\\ {\sigma }_1{u}_{l1}& {\sigma }_2{u}_{l2}& \dots & {\sigma }_{\lambda }{u}_{l\lambda }& 0& 0\end{array}\right)$$

$$\mathbb{V}=\left(\begin{array}{cccc}{v}_{11}& v_{12}& \dots & v_{1N}\\ v_{21}& v_{22}& \dots & v_{2N}\\ ⋮& ⋮& \ddots & ⋮\\ v_{N1}& v_{N2}& \dots & v_{NN}\end{array}\right)$$

$$\Sigma {\mathbb{V}}^T=\left(\begin{array}{cccc}{\sigma }_1{v}_{11}& {\sigma }_1{v}_{21}& \dots & {\sigma }_1{v}_{N1}\\ {\sigma }_2{v}_{12}& {\sigma }_2{v}_{22}& \dots & {\sigma }_2{v}_{N2}\\ ⋮& ⋮& \ddots & ⋮\\ {\sigma }_{\lambda }{v}_{1\lambda }& {\sigma }_{\lambda }{v}_{2\lambda }& \dots & {\sigma }_{\lambda }{v}_{N\lambda }\\ 0& 0& \dots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \dots & 0\end{array}\right)$$

$$\mathbb{U}=\left(\begin{array}{cccc}{u}_{11}& u_{12}& \dots & u_{1l}\\ u_{21}& u_{22}& \dots & u_{2l}\\ ⋮& ⋮& \ddots & ⋮\\ u_{l1}& u_{l2}& \dots & u_{ll}\end{array}\right)$$

$$\{\begin{array}{rl}{\sigma }_1{v}_{i1}\cdot u_{11}+\cdots & +{\sigma }_{\lambda }{v}_{i\lambda }{u}_{1\lambda }=d_{1i}\\ ⋮& \\ {\sigma }_1{v}_{i1}\cdot u_{l1}+\cdots & +{\sigma }_{\lambda }{v}_{i\lambda }{u}_{l\lambda }=d_{li}\end{array}$$

$$|{m_{i}}^{{}^{\prime}}|=\left\{\begin{array}[]{ll}l[3max(\kappa_{N},\kappa_{l})+4\kappa_{d}+3\kappa_{N}+3\kappa_{l}+3]&\mbox{, if }max(\kappa_{N},\kappa_{l})+2\kappa_{1}+2\kappa_{2}>80.\\ \\ l(80+\kappa_{N}+\kappa_{l}+1)&\mbox{, else.}\\ \end{array}\right.$$

$$d_{ik}^{\ast }=N\cdot d_{ik}^{{}^{\prime }}-\sum _{m=1}^N{d}_{im}^{{}^{\prime }}$$

$${\delta }_i=\sqrt{\frac{e_{ii}^{\ast }}{N-1}}$$

$$\begin{array}{rl}{d}_{ik}^{\ast }& =N\cdot d_{ik}^{{}^{\prime }}-\sum _{m=1}^N{d}_{im}^{{}^{\prime }}\\ & =(N{d}_{ik}-\sum _{m=1}^N{d}_{im})+(N{z}_{ik}-\sum _{m=1}^N{z}_{im})W+(N{r}_{ik}-\sum _{m=1}^N{r}_{im})S\end{array}$$

$$\begin{array}{rl}{e}_{ij}& =\sum _{k=1}^N{d}_{ik}^{\ast }\cdot d_{jk}^{\ast }\\ & =\sum _{k=1}^N\widehat{d_{ik}}\cdot \widehat{d_{jk}}+\sum _{k=1}^N[(\widehat{z_{ik}}\widehat{d_{jk}}+\widehat{z_{jk}}\widehat{d_{ik}})W+\widehat{z_{ik}}\widehat{z_{jk}}{W}^2]\\ & +S\sum _{k=1}^N[(\widehat{r_{ik}}\widehat{d_{jk}}+\widehat{r_{jk}}\widehat{d_{ik}})+(\widehat{z_{ik}}\widehat{r_{jk}}+\widehat{z_{jk}}\widehat{r_{ik}})W+\widehat{r_{ik}}\widehat{r_{jk}}S]\end{array}$$

Full text

1 Introduction

With the prosperous development of communication and computation technologies, the Internet of Things (IoT), which allows the physical objects to be sensed, accessed and controlled remotely through the network infrastructure, is no longer a fantasy nowadays. The big advantage of IoT is that with the data analysis on the huge amount of information collected from the physical world, the server is capable of making more accurate and optimal decisions which would produce considerable benefits. It is estimated that the global IoT market will be 14.4 trillion dollars by 2022 [1], the potential economic impact of IoT would be 3.9 to 11.1 trillion dollars by 2025 [2], and the number of devices connected to the Internet would be about 50 billion by 2020 [3]. It could be expected that rather than being conducted on the cloud or inside the intranet of companies, the data analysis work would be performed everywhere and anytime in the future due to the ubiquitous IoT. As the amount of data analysis tasks increases in IoT, the singular value decomposition (SVD), which is widely used in different data analysis applications [4][5][6][7][8], will be performed frequently. However, the traditional way of performing SVD, i.e. calculating the SVD in central server, may not be practical in future IoT due to the huge amount of IoT devices. If all the data is transmitted to a central server for computation, it would lead to considerable computation and communication resource consumption of the server, which would further severly impact the quality of service (QoS) of IoT applications.

To ease the burden of the IoT server and guarantee the QoS, a new technique called fog computing, which is proposed by Cisco [9], is suitable to be applied. The main idea of fog computing is to provide storage, computing and networking services between environmental devices and the central server. The fog devices which are in close proximity to end devices normally possess considerable storage and computation resource. With the equipped resource, the fog devices could process the collected data locally so as to loose the workload of the server. In specific, there are three tiers in the fog computing architecture: environmental tier, edge tier and central tier. In the environmental tier, there are billions of heterogeneous IoT devices collecting and uploading information of the physical world, e.g. medical sensors in eHealth and mobile phone of each people. The data collected by IoT devices will be transmitted to the edge tier. The distribution of fog devices in the edge tier are hierarchical which is a characteristic inherited from the traditional network architecture. For example, the switchers of a local area network could function as the first layer fog devices and the gateways which manage those switchers could serve as the second layer fog devices. The fog devices in the edge tier could perform the application-specific operations on received data locally and send the results to the server in the central tier. Owing to the processing of fog devices, the volume of data sent to server could be reduced to a large extent. Since the fog devices are spread in a highly distributed environment, it is impractical for the government or an institution which owns the central server to provide and maintain all those fog devices. Therefore, it is reasonable to assume that the fog devices would be supplied by third parties.

Under the context of fog computing, one could perform the SVD operations on the fog devices. However, another problem which would appear is the privacy issue. The third parties which control the fog devices may not be trustworthy while in many IoT applications, the data collected from the environment is considered as private by the IoT application users, e.g. the vital signs in eHealth, the location and speed of vehicles, and the power usage in a smart grid. Performing the SVD on plaintext with fog devices is infeasible if the privacy is a primary concern from the perspective of data owners. Therefore, how to take advantage of fog computing to locally process data in a privacy-preserving way is a challenging issue.

In this paper, we propose a flexible fog computing framework for performing SVD with privacy preserved. The homomorphic encryption technique called Paillier encryption [10] is applied to protect the data privacy. The framework is designed to be capable of supporting different applications based on the SVD computation. The main contributions of this paper are three-fold.

• •

First, to perform data analysis for IoT applications, we propose a fog computing framework for SVD computation to ease the burden of server. Since the computation is performed in the fog devices which may not be trustable, the framework achieves the privacy-preserving SVD computation.

• •

Second, there is only one communication round between the data providers and data processors in our work while most of the existing works require iterative communications, which brings heavy overhead.

• •

Third, besides the framework for basic SVD operation, three applications are introduced in details to demonstrate the flexibility of the framework. It has been shown that the proposed framework could flexibly adapt to different applications with slight adjustments or several extra procedures.

The reminder of this paper is organized as follow. In Section 2, the preliminaries of our scheme are introduced. The system model, security requirements and design goals are described in Section 3. In Section 4, the proposed framework is presented in details. The security analysis and performance evaluation are discussed in Section 5 and 6. Three applications based on the proposed privacy-preserving SVD framework are illustrated in Section 7. In Section 8, we discuss the related work, and finally conclude our current work in Section 9.

2 Preliminaries

In this section, the Paillier Cryptosystem [10] and Singular Value Decomposition [11] which are the basis of the proposed framework are reviewed.

2.1 Paillier Cryptosystem

The Paillier Cryptosystem enables the addition and multiplication operations on plaintext through the specific linear algebraic manipulation conducted on the ciphertext. This property is extensively desired in many privacy-preserving applications [12, 13, 14]. In this paper, this feature allows fog devices to process the user data in encrypted form without leaking the data content. The Paillier Cryptosystem comprises three phases: key generation, encryption and decryption.

• •

*Key Generation: *Given one security parameter $\kappa$, generate two large prime numbers $p$, $q$, where $|p|=|q|=\kappa$. Then compute the $\mathcal{RSA}$ modulus $n=pq$, $\lambda=lcm(p-1,q-1)$ and choose a generator $g\in\mathbb{Z}_{n^{2}}^{\ast}$. Define the function $L(u)=\frac{u-1}{n}$ and calculate $\mu=(L(g^{\lambda}\bmod n^{2}))^{-1}\bmod n$. Then PK $=(n,g)$ is published as the public key and SK $=(\lambda,\mu)$ is kept as the corresponding private key.

• •

*Encryption: *Given a message $m\in\mathbb{Z}_{n}$, randomly choose a number $r\in\mathbb{Z}_{n}^{\ast}$, the ciphertext could be calculated as $c=E(m,r)=g^{m}\cdot r^{n}\bmod n^{2}$.

• •

*Decryption: *Given the ciphertext $c\in\mathbb{Z}_{n^{2}}^{\ast}$, the corresponding plaintext could be recovered as $m=D(c)=L(c^{\lambda}\bmod n^{2})\cdot\mu\bmod n$. Note that, the Paillier Cryptosystem is provably secure against chosen plaintext attack, and the correctness and security can be referred to [10].

The homomorphic property of the Paillier Cryptosystem utilized in this work is: $E(m_{1},r_{1})\cdot E(m_{2},r_{2})=E(m_{1}+m_{2},r_{1}\cdot r_{2})$.

2.2 Singular Value Decomposition

SVD is a powerful and popular matrix factorization tool that underlies plenty of useful applications, e.g. abnormal detection [4, 5], recommendation system [6][7] and data compression [8]. Let $\mathbb{A}$ be a $l\times N$ matrix, the SVD of $\mathbb{A}$ is of the form $\mathbb{U}\Sigma\mathbb{V}^{T}$ where $T$ means conjugate transpose, $\mathbb{U}$ is a $l\times l$ unitary matrix, $\Sigma$ is a $l\times N$ rectangular diagonal matrix with non-negative diagonal values, and $\mathbb{V}$ is an $N\times N$ unitary matrix. The non-negative diagonal entries of $\Sigma$ are the singular values of matrix $\mathbb{A}$ while the columns of $\mathbb{U}$ and $\mathbb{V}$ are known as the left-singular vectors and right-singular vectors of $\mathbb{A}$. Note that we only consider real matrix entries in this paper, thus the conjugate transpose $T$ could be simply regarded as transpose.

Another widely used matrix factorization tool is the eigenvalue decomposition. Unlike SVD which could be applied to any $l\times N$ matrix, the eigenvalue decomposition is less general and could only be performed on square matrix. However, the two kinds of tool are closely related shown as follow:

[TABLE]

It is obvious that the left-singular vectors $\mathbb{U}$ is the eigenvectors of $\mathbb{A}\cdot\mathbb{A}^{T}$, the right-singular vectors $\mathbb{V}$ is the eigenvectors of $\mathbb{A}^{T}\cdot\mathbb{A}$ and the singular values $\Sigma$ are the square root of the eigenvalues of $\mathbb{A}\cdot\mathbb{A}^{T}$ and $\mathbb{A}^{T}\cdot\mathbb{A}$. We will show that the above relation could be utilized to achieve the privacy-preserving SVD in the later sections.

3 System Model, Security Requirements and Design Goals

In this section, we describe the system model, discuss the security requirements and identify the design goals on privacy-preserving SVD.

3.1 System Model

In this work, we mainly focus on how to utilize the fog computing to compute the SVD of the data uploaded by environmental devices with privacy preserved. Specifically, there are four categories of entity in the system model, namely server, first layer fog device, second layer fog device and environmental device as shown in Fig. 1.

Server: server is a fully trustable entity located in the central tier. It is responsible for initializing the whole system and distributing key materials to others. The other operations the server may conduct are application-specific. Serval examples will be given in Section 7.

Environmental Device (ED): EDs are the devices distributed in the environmental layer of IoT environment. The analysis on the data uploaded by EDs could enable better decision-making.

First Layer Fog Device (FD): FDs are the fog devices which communicate with EDs directly. FDs process the collected data and upload the results to the second layer fog devices.

Second Layer Fog Device (SD): SDs are the fog devices which communicate with FDs. Compared to FDs, SDs are closer to the server and do not contact with EDs directly. In the proposed framework, there are three SDs playing different roles for SVD operation. One of them is responsible for decrypting the messages from FDs. The other two are in charge of decomposing $\mathbb{A}\cdot\mathbb{A}^{T}$ and $\mathbb{A}^{T}\cdot\mathbb{A}$. We denote the one for decryption, decomposing $\mathbb{A}\cdot\mathbb{A}^{T}$ and decomposing $\mathbb{A}^{T}\cdot\mathbb{A}$ as SDd, SDu and SDv respectively.

The basic workflow of our model is: the server first initializes the whole system and distributes the key materials or secrets to other entities accordingly. After initialization, EDs start to collect and upload application-specific data as required. Each data transmitted to FDs is encrypted. After FDs gather the data from EDs, FDs will randomize the collected data and send the results to SDd. Upon decrypting the encrypted data from FDs, SDd will perform further operations on the plaintext and send the outcomes to SDu and SDv respectively. With the messages from SDd, SDu and SDv could recover $\mathbb{A}\mathbb{A}^{T}$ and $\mathbb{A}^{T}\mathbb{A}$, and conduct the eigenvalue decomposition accordingly. Finally, the SVD of data collection is split into two parts which are held by SDu and SDv seperately.

3.2 Security Requirements

Security is fundamental for the effectiveness of proposed framework. In this work, the server and EDs are assumed to be trustable. The fog devices, i.e. FDs and SDs, are assumed to be honest-but-curious [15, 16] which means they will follow the specified procedures faithfully while being curious about the uploaded data. In addition, FDs and SDs are assumed not to collude with each other. The non-collusion assumption could be realized similarly as the EigenTrust scheme [17]. Briefly speaking, for each SVD computation, the server chooses fog devices based on distributed hash table. Due to the large number of fog devices, it is infeasible for the device providers to determine whether they would be selected for the same computation and negotiate for collusion in advance. Based on the above assumptions, the confidentiality as the security requirement should be fulfilled, i.e. even FDs and SDs process the collected data, they could not learn anything about the actual value of data. For authenticity and integrity, since there are many existing signature schemes for them, e.g. Boneh-Lynn-Shacham (BLS) short signature [18], this work focuses on confidentiality.

3.3 Design Goals

According to the aforementioned system model and security requirements, the goal is to design a flexible fog computing framework for privacy-preserving SVD computation. Specifically, the proposed framework should achieve the following objectives.

• •

The confidentiality should be guaranteed in the proposed framework. All the user data contained in the transmitted messages should be protected. The processing in fog devices should not leak data privacy.

• •

The framework should be flexible enough to be adopted by different applications. Instead of being the ultimate goal, SVD is the basis or initial step of many applications, which means the further procedures after SVD could be quite different for various scenarios. Therefore, the design of the framework should consider the flexibility such that the results of SVD could be further utilized to achieve the final purposes of different applications.

4 The Proposed Framework

In this section, the proposed framework for SVD computation is presented in details. The framework is composed of five phases: system initialization, data collection, data randomization, pre-computation and eigenvalue decomposition.

4.1 System Initialization

The server is the trustable entity which bootstraps the whole system. Assume the amount of users supported by the system is $N$, each user data is $l$-dimensional and the range for each dimension value is [[math], $d$] where $d$ is a constant. The system parameters are $\kappa$, $\kappa_{1}$, $\kappa_{2}$ and $\kappa_{3}$. Let $|\bullet|$ denote the bit length of $\bullet$. Given the parameter $\kappa$, the server calculates the public key for Paillier Cryptosystem PK: $(n=pq,g)$, and the corresponding private key SK: $(\lambda,\mu)$, where $p$, $q$ are two large primes with $|p|=|q|=\kappa$. Given the parameters $\kappa_{1}$, $\kappa_{2}$ and $\kappa_{3}$, let $t=2^{\kappa_{1}}$, the server randomly chooses two coprime integers $W$ and $S$ such that $W>max(N,l)\cdot{d^{2}}$ and $S>max(N,l)\cdot{(d^{2}+2tWd+t^{2}W^{2})}$, where $|W|=\kappa_{2}$ and $|S|=\kappa_{3}$. Then, the server chooses one superincreasing sequence $\vec{\mathbf{a}}=(a_{1}=1,a_{2},\cdots,a_{l})$ such that $\sum_{j=1}^{i-1}a_{j}\cdot(d+tW+tS)<a_{i}$ for $i=2,\cdots,l$ and $\sum_{i=1}^{l}a_{i}\cdot(d+tW+tS)<n$. Finally, the server publishes $\{n,g,\vec{\mathbf{a}}\}$ as public parameters, sends $(\lambda,\mu)$ to SDd as secret, and sends $(W,S)$ to FDs, SDu and SDv as secret respectively.

4.2 Data Collection

In the environmental tier, EDs collect and upload the application-specific $l$-dimensional data $(d_{1},\cdots,d_{l})^{T}$. The data from $N$ EDs could form the data matrix $\mathbb{A}$.

[TABLE]

To compute the SVD of matrix $\mathbb{A}$ with privacy preserved is the goal of this work. The $i$th column of matrix $\mathbb{A}$ $(d_{1i},\cdots,d_{li})^{T}$ is the data from the $i$th device EDi. To upload the data, EDi performs the following steps:

• •

*Step-1. *Utilize the superincreasing $\vec{\mathbf{a}}$ to compute

[TABLE]

• •

*Step-2. *Choose a random number $r_{i}\in\mathbb{Z}_{n}^{\ast}$ and compute

[TABLE]

• •

*Step-3. *Send the encrypted data $C_{i}||ED_{i}$ to the FD which communicates with it.

4.3 Data Randomization

For each FD, it will perform the following steps to randomize the received data.

• •

*Step-1. *For the $i$th data $C_{i}$, FD chooses $2\cdot l$ random numbers which are $(z_{1i},\cdots,z_{li})^{T}$ and $(r_{1i},\cdots,r_{li})^{T}$ from the range $[1,t]$. Then FD computes $rz=\sum_{k=1}^{l}a_{k}\cdot(z_{ki}\cdot W+r_{ki}\cdot S)$.

• •

*Step-2. *FD randomizes $C_{i}$ as

[TABLE]

• •

Step-3. FD sends the randomized data ${C_{i}}^{{}^{\prime}}$ to SDd.

4.4 Pre-computation

Upon receiving $N$ data from FDs, SDd will perform the following steps to compute the randomized $\mathbb{A}\mathbb{A}^{T}$ and $\mathbb{A}^{T}\mathbb{A}$.

• •

Step-1. For each ${C_{i}}^{{}^{\prime}}$, SDd will utilize the secret key $(\lambda,\mu)$ to decrypt it and get the aggregation of randomized data

[TABLE]

• •

Step-2. Through running the Algorithm 1, SDd could recover the randomized value for each dimension of data $i$.

• •

Step-3. From each ${m_{i}}^{{}^{\prime}}$, SDd could get an $l$-dimensional randomized data. In total, SDd could get the randomized data matrix

[TABLE]

The $(i,j)$th entry of $\mathbb{A}^{{}^{\prime}}$ is $d_{ij}^{{}^{\prime}}=d_{ij}+z_{ij}\cdot W+r_{ij}\cdot S$. Then SDd simply computes $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$ and $(\mathbb{A}^{{}^{\prime}})^{T}\cdot\mathbb{A}^{{}^{\prime}}$, and sends the two resulting matrices to SDu and SDv respectively.

The correctness of Algorithm 1. In Algorithm 1, $X_{l}={m_{i}}^{{}^{\prime}}=a_{1}\cdot(d_{1i}+z_{1i}\cdot W+r_{1i}\cdot S)+\cdots+a_{l-1}\cdot(d_{(l-1)i}+z_{(l-1)i}\cdot W+r_{(l-1)i}\cdot S)+a_{l}\cdot(d_{li}+z_{li}\cdot W+r_{li}\cdot S)]\bmod n$. Since the data value for each dimension is in the range of [[math], $d$], and $z_{ki}$ and $r_{ki}$ are chosen from $[1,t]$, we have

[TABLE]

Therefore, $X_{l-1}=X_{l}\bmod a_{l}=a_{1}\cdot(d_{1i}+z_{1i}\cdot W+r_{1i}\cdot S)+\cdots+a_{l-1}\cdot(d_{(l-1)i}+z_{(l-1)i}\cdot W+r_{(l-1)i}\cdot S)$, and

[TABLE]

Similarly, it can be proved that $t_{ak}=d_{ki}+z_{ki}\cdot W+r_{ki}\cdot S$, for $k=1,\cdots,l-1$.

4.5 Eigenvalue Decomposition

4.5.1 SDu:

When receiving $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$, SDu will perform the following steps to compute the left part of the SVD for matrix $\mathbb{A}$, i.e. matrix $\mathbb{U}$ and $\Sigma$.

• •

Step-1. For each entry ${e_{u}}^{{}^{\prime}}$ of $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$, SDu derandomizes the entry as follow:

[TABLE]

The result $e_{u}$ is the corresponding entry of matrix $\mathbb{A}\cdot\mathbb{A}^{T}$.

• •

Step-2. After SDu recovers the matrix $\mathbb{A}\cdot\mathbb{A}^{T}$, it performs eigenvalue decomposition for matrix $\mathbb{A}\cdot\mathbb{A}^{T}$ and gets the matrix $\mathbb{U}$ and $\Sigma$.

4.5.2 SDv:

Similar as SDu, SDv will derandomize the entries of received $(\mathbb{A}^{{}^{\prime}})^{T}\cdot\mathbb{A}^{{}^{\prime}}$ to recover the matrix $\mathbb{A}^{T}\cdot\mathbb{A}$. Then SDv performs the eigenvalue decomposition on $\mathbb{A}^{T}\cdot\mathbb{A}$ to get the right part of $\mathbb{A}$’s SVD, i.e. matrix $\mathbb{V}$ and $\Sigma$.

By now, the SVD of $\mathbb{A}$ has been seperately held by SDu and SDv.

The correctness of derandomization The $(i,j)$th entry ${e_{u}}^{{}^{\prime}}_{ij}$ of $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$ is implicitly formed as

[TABLE]

Since

[TABLE]

we have ${e_{u}}^{{}^{\prime}}_{ij}\bmod S=\sum_{k=1}^{N}d_{ik}d_{jk}+\sum_{k=1}^{N}[(z_{ik}d_{jk}+z_{jk}d_{ik})W+z_{ik}z_{jk}W^{2}]$. Also, we have $\sum_{k=1}^{N}d_{ik}d_{jk}<Nd^{2}<W$. Thus, $({e_{u}}^{{}^{\prime}}_{ij}\bmod S)\bmod W=\sum_{k=1}^{N}d_{ik}d_{jk}$ which is the $(i,j)$th entry of matrix $\mathbb{A}\cdot\mathbb{A}^{T}$.

Similarly, for the $(i,j)$th entry ${e_{v}}^{{}^{\prime}}_{ij}$ of matrix $(\mathbb{A}^{{}^{\prime}})^{T}\cdot\mathbb{A}^{{}^{\prime}}$, we have $({e_{v}}^{{}^{\prime}}_{ij}\bmod S)\bmod W=\sum_{k=1}^{l}d_{ki}\cdot d_{kj}$ which is the $(i,j)$th entry of matrix $\mathbb{A}^{T}\cdot\mathbb{A}$.

5 Security Analysis

In this section, the security properties of the proposed framework are analysed. As mentioned in the security requirements of Section 3, the participants of the framework will faithfully follow the defined work procedures while being curious about the user data. Thus, we first analyze the ability of each participant to learn private data under normal operations, i.e. the probability of leaking privacy when following legal processes. Then, the possible extra operations, denoted as potential attacks, which could be conducted by certain participants to snoop data are analyzed. The resistance of the framework against those attacks is discussed and the principles for system configuration are demonstrated.

5.1 Privacy Leakage Probability under Normal Operations

5.1.1 EDs:

In the proposed framework, each ED will encrypt its data with Paillier Cryptosystem before uploading. Thus, each ED could only know the plaintext of its own message and learn nothing about the other EDs’ data.

5.1.2 FDs:

The messages collected by the FDs are all Paillier ciphertext. Since the FDs do not have the private key for decrypting messages and it is assumed that there is no collusion among fog devices, the user data privacy is protected by the Paillier Cryptosystem no matter what operations are performed on the ciphertext by FDs.

5.1.3 SDd:

SDd is the only fog device which has the private key for decryption. However, the plaintexts SDd could get are the randomized data, i.e. ${d_{ij}}^{{}^{\prime}}=d_{ij}+z_{ij}W+r_{ij}S,$ for $i=1,\cdots,l$ and $j=1,\cdots,N$. Since SDd has no idea about the value of $z_{ij},r_{ij},W$ and $S$, it could not gain any knowledge of the original data.

5.1.4 SDu:

Through derandomizing $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$, SDu obtains $\mathbb{A}\cdot\mathbb{A}^{T}$ and further gets $\mathbb{U}$ and $\Sigma$. Since $\mathbb{A}=\mathbb{U}\Sigma\mathbb{V}^{T}$, SDu needs to find the correct unitary matrix $\mathbb{V}$ to determine $\mathbb{A}$. Since there are infinite unitary matrices, SDu could not recover original data matrix $\mathbb{A}$ with only $\mathbb{U}$ and $\Sigma$.

5.1.5 SDv:

Similar as SDu, SDv could not recover data matrix $\mathbb{A}$ with only $\mathbb{V}$ and $\Sigma$.

Based on the above analysis, the data privacy is totally preserved in the proposed framework when the participants follow the defined procedures. In the following, the possible extra computations performed by participants to discover private data are considered.

5.2 Potential Attacks

For EDs and FDs, since they only have encrypted user data, they could not get any meaningful information no matter what operations they perform on the ciphertext. Therefore, we mainly discuss the potential attacks from SDd, SDu and SDv in this part.

$\bullet$ SDd: As mentioned above, the information SDd gets is the randomized data ${d_{ij}}^{{}^{\prime}}=d_{ij}+z_{ij}W+r_{ij}S,$ for $i=1,\cdots,l$ and $j=1,\cdots,N$. What SDd needs to do is to find the value of $S$ and $W$ and recover the original data as

[TABLE]

Since ${d_{ij}}$ is mixed with the random combination of $S$ and $W$, it is infeasible for SDd to determine $S$ and $W$ without additional information. Therefore, we consider the situations in which SDd knows some of the user data. With the knowledge of user data, the possible operations SDd could do are as follows:

• •

Step-1. For each known data, SDd converts the corresponding randomized data to the form $zW+rS$ by computing ${d}^{{}^{\prime}}-d$. Let $LC$ denote the set of converted data and $LC_{i}=z_{i}W+r_{i}S$ denote the $i$th element of $LC$.

• •

Step-2. SDd performs the brute force attack, i.e. tries all possible $S$. For each try, SDd performs (modulo $S$) operation on each $LC_{i}$. Then SDd computes the greatest common divisor (GCD) of the resulting set. If the GCD is larger than 1, it is the value of $W$ and the currently selected $S$ is the correct $S$.

The rationale behind this attack is: the probability of $k$ randomly chosen integers being coprime is $\frac{1}{\varsigma(k)}$, where $\varsigma(x)$ is Riemann zeta function [19]. When $k$ is large, the probability that they are not coprime is negligible. Thus, after the modulo operations on $LC$, only when the chosen $S$ is correct, the elements of the resulting set are of the form $zW$ and have a GCD larger than 1, which is $W$.

Note that, in some cases, SDd could still form a set, in which the elements are of the form $zW+rS$ with high probability, even it does not know any user data. For example, if the data matrix is sparse, most of the randomized data is already of the desired form. Another case is that data range is not large enough compared to the amount of data, SDd could compute $DV=d_{ij}^{{}^{\prime}}-d_{i^{{}^{\prime}}j^{{}^{\prime}}}^{{}^{\prime}}$ for all possible pairwise combinations $(d_{ij}^{{}^{\prime}},d_{i^{{}^{\prime}}j^{{}^{\prime}}}^{{}^{\prime}})$ and some of the resulting $DV$s will be of the desired form. For those cases, SDd could perform the brute force attack.

Parameter Selection. To resist the brute force attack in the possible cases, $|S|$, i.e. $\kappa_{3}$, should be at least equal to 80. Moreover, SDd could compute $LDV=LC_{i}-LC_{j}$ for all possible combinations $(LC_{i},LC_{j})$. If certain combinations have the same $zW$ inside, those resulting $LDV$s would be of the form $(r_{i}-r_{j})S$. SDd could learn $S$ efficiently by computing the GCD of those $LDV$s even when $|S|\geq 80$. To avoid the case, the randomly chosen $z_{ij}$ should be different with each other with high probability. The $z_{ij}$ is chosen from the range $[1,t]$, and the total number of $z_{ij}$ is $l\cdot N$. According to the generalized birthday problem [20], the probability of at least two chosen $z_{ij}$ match is $1-\exp^{\frac{-(lN)^{2}}{2t}}$. Thus, the probability of no match is $\exp^{\frac{-(lN)^{2}}{2t}}$ and the parameter $\kappa_{1}$ which determines $t$ could be selected accordingly. Note that if FDs could cooperatively choose the set of $z_{ij}$ such that there is no match, then the range $t$ only needs to be larger than $l\cdot N$.

$\bullet$ SDu: SDu could get $\mathbb{U}$, $\Sigma$ and $\mathbb{U}\Sigma$:

[TABLE]

[TABLE]

[TABLE]

where $\lambda$ is the rank of $\mathbb{A}$. The purpose of SDu is to find the matrix $\mathbb{V}$:

[TABLE]

Note that the first $\lambda$ elements of each row in $\mathbb{V}$ correspond to a column of $\mathbb{A}$. If SDu knows the left $\lambda$ columns of $\mathbb{V}$, it could recover the original data. Since there are infinite unitary matrices, SDu could not determine the correct $\mathbb{V}$ if it has no additional information. Thus, we assume that SDu could get $N^{{}^{\prime}}$ original data in some cases. Note that using one data, i.e. one column of $\mathbb{A}$, could form $l$ equations for the same row of $\mathbb{V}$. Solving the equations from one user data, SDu could determine the first $\lambda$ elements of that row.

When $N^{{}^{\prime}}<N-1$, since each row of $\mathbb{V}$ is linearly independent with each other, obtaining one row does not help to learn the other rows. Thus, SDu could not utilize the known data to learn the rest unknown data.

When $N^{{}^{\prime}}=N-1$, SDu could determine the first $\lambda$ elements of $(N-1)$ rows of $\mathbb{V}$, then the first $\lambda$ elements of the last unknown row could be determined due to $\sum_{i=1}^{N}{v_{ij}^{2}}=1$. The last unknown user data could be recovered accordingly.

$\bullet$ SDv: Similar as SDu, the matrix which could be utilized by SDv is $\Sigma\mathbb{V}^{T}$:

[TABLE]

where $\lambda$ is the rank of $\mathbb{A}$. The purpose of SDv is to find the matrix $\mathbb{U}$:

[TABLE]

Different from the case of SDu, the first $\lambda$ elements of each row in $\mathbb{U}$ correspond to a row of $\mathbb{A}$, i.e. the data value of a certain dimension from all users. If SDv knows the left $\lambda$ columns of $\mathbb{U}$, it could recover the original data. Similarly, we assume that SDv could get $N^{{}^{\prime}}$ linearly independent user data in some cases. We have ”linearly independent” here because linearly dependent data would not produce new linearly independent equations. Thus, only the number of linearly independent data matters. Each user data $\vec{d}_{i}$, i.e. one column of $\mathbb{A}$, could form one equation for each row of $\mathbb{U}$:

[TABLE]

When $\lambda<l$, if $N^{{}^{\prime}}=\lambda$, SDv could form $\lambda$ linearly independent equations for each row of $\mathbb{U}$. Then through solving equations, SDv could determine the left $\lambda$ columns of $\mathbb{U}$ and thus recover the whole $\mathbb{A}$ which contains the other unknown user data. On the other hand, if $N^{{}^{\prime}}<\lambda$, SDv could not recover the other unknown linearly independent data due to the lack of enough linearly independent equations. However, for the data which is linearly dependent with the known data, SDv could recover them because the columns of $\Sigma\mathbb{V}^{T}$ have the same linear relationship as those existing among user data.

When $\lambda=l$, there is an additional condition for solving the equations of $\mathbb{U}$, i.e. $\mathbb{U}$ is a unitary matrix. Specifically, the $l$ rows of $\mathbb{U}$ could be regarded as the coordinate axis of $l$-dimensional space whose rotation degree of freedom is $l-1$. For each linearly independent data known to SDv, the rotation degree of freedom of the coordinate axis reduces by 1. Therefore, if $N^{{}^{\prime}}=l-1$, the rotation degree of freedom of the coordinate axis reduces to 0, i.e. the coordinate axis is fixed. Moreover, since $\sum_{j=1}^{l}{u_{ij}^{2}}=1$, each row of $\mathbb{U}$ could be seen as a point locating on the unit sphere of $l$-dimension. Thus, the intersection points between the fixed coordinate axis and the $l$-dimensional unit sphere is the solution of $\mathbb{U}$.

Based on the above analysis, the proposed framework could resist the potential attacks launched by SDd through properly choosing $z_{ij}$ and $S$. For SDu, only when $(N-1)$ user data is obtained, it could learn the value of the last unknown data. For SDv, if $\lambda<l$, the framework could resist not more than $(\lambda-1)$ linearly independent user data leakage and it could resist not more than $(\lambda-2)$ linearly independent user data leakage if $\lambda=l$.

6 Performance Evaluation

In this section, we evaluate the performance of the proposed fog computing framework in terms of the capacity and efficiency. The capacity demonstrates the number of required ciphertexts for different matrix sizes while the efficiency indicates the computational complexity and communication overhead of the framework.

6.1 Capacity

In the proposed framework, the aggregated randomized data is of the form ${m_{i}}^{{}^{\prime}}=\sum_{k=1}^{l}a_{k}\cdot(d_{ki}+z_{ki}\cdot W+r_{ki}\cdot S)$. To guarantee the aggregated data could be recovered correctly through decryption, ${m_{i}}^{{}^{\prime}}$ should be less than $n$, i.e. the constraint $\sum_{k=1}^{l}a_{k}\cdot(d+tW+tS)<n$ must be fulfilled. At the same time, the superincreasing sequence $\vec{\mathbf{a}}$ needs to meet the constraint: $\sum_{j=1}^{i-1}a_{j}\cdot(d+tW+tS)<a_{i}$ for $i=2,\cdots,l$. Moreover, in order to successfully derandomize the data, $W$ and $S$ need to fulfill: $W>max(N,l)\cdot{d^{2}}$ and $S>max(N,l)\cdot{(d^{2}+2tWd+t^{2}W^{2})}$. To resist the potential attacks from SDd, the value of $t$ should be chosen based on $N$ and $l$, and $\kappa_{3}$ should not be less than 80.

Let $\kappa_{N},\kappa_{l}$ and $\kappa_{d}$ denote the bit length of $N$, $l$ and $d$ respectively. To resist the attacks from SDd, $\kappa_{1}$ should be larger than $\kappa_{N}+\kappa_{l}$. For simplicity, assume that FDs could cooperatively select the $z_{ij}$ such that no match happens. Then $\kappa_{1}=\kappa_{N}+\kappa_{l}+1$ is enough. To meet $W>max(N,l)\cdot{d^{2}}$, we have $\kappa_{2}>max(\kappa_{N},\kappa_{l})+2\kappa_{d}$. Then due to $S>max(N,l)\cdot{(d^{2}+2tWd+t^{2}W^{2})}$, $\kappa_{3}>max(\kappa_{N},\kappa_{l})+2\kappa_{1}+2\kappa_{2}>3\cdot max(\kappa_{N},\kappa_{l})+4\kappa_{d}+2\kappa_{N}+2\kappa_{l}+2$. For the superincreasing sequence $\vec{\mathbf{a}}$, $|a_{2}|>\kappa_{3}+\kappa_{1}$ and $|a_{3}|>2(\kappa_{3}+\kappa_{1})$. It is easy to find that $|a_{i}|>(i-1)(\kappa_{3}+\kappa_{1})$ and $|\sum_{k=1}^{l}a_{k}\cdot(d+tW+tS)|>l(\kappa_{3}+\kappa_{1})$. Thus, the bit length of aggregated data:

[TABLE]

It is obvious that the data dimension has a great influence on the aggregated data length. Given different $\kappa_{d}$ and $l$, the number of users which one ciphertext with $|n|=1024$ could support is evaluated as shown in Fig. 2(a).

From Fig. 2(a), it could be seen that the increase of dimensionality could dramatically decrease the number of users which one ciphertext could support, while the impact of data range $d$ is not that significant. One ciphertext could support large number of users with low dimensional data, e.g. $2^{37}$ users with $4$-dimensional data and $2^{15}$ users with $8$-dimensional data. To support higher dimensional data for the same amount of users, each ED needs to use multiple ciphertexts to aggregate data, e.g. to support $2^{15}$ users with 16-dimensional data needs 2 ciphertexts each of which aggregates 8 dimensions. Given different $l$ and $\kappa_{N}$, the number of required ciphertexts with $|n|=1024$ is evaluated in Fig. 2(b) and Fig. 2(c) respectively. It could be seen that each ED needs to use $O(l\cdot log(N))$ ciphertexts for uploading data.

6.2 Efficiency

As analyzed above, each ED may need more than one Paillier ciphertext for aggregating user data. Let $N_{C}$ denote the number of required ciphertexts for each ED. In the following, the computational complexity and communication overhead of the proposed framework are analyzed.

Computational Complexity: Since the crypto-operations are much heavier than the computations on plaintext, the amount of crypto-operations is the main concern in this part. The overall crypto-operations performed by each entity in the procedures of proposed framework are shown as follow.

$\bullet$ ED: each encryption needs $2$ exponentiation and $1$ multiplication in $\mathbb{Z}_{n^{2}}$. The overall crypto-operations conducted by an ED is $2\cdot N_{C}$ exponentiation and $N_{C}$ multiplication in $\mathbb{Z}_{n^{2}}$.

$\bullet$ FD: each randomization needs $1$ exponentiation and $1$ multiplication in $\mathbb{Z}_{n^{2}}$. Assuming the number of EDs communicating with the FD is $N_{e}$, the overall crypto-operations performed by a FD is $N_{e}\cdot N_{C}$ exponentiation and multiplication in $\mathbb{Z}_{n^{2}}$ respectively.

$\bullet$ SDd: to decrypt one ciphertext, SDd needs to perform $1$ exponentiation operation in $\mathbb{Z}_{n^{2}}$. After decryption, the other operations are conducted on plaintexts and those cost is negligible compared to decryption. The overall crypto-operations conducted by the SDd is $N\cdot N_{C}$ exponentiation in $\mathbb{Z}_{n^{2}}$.

Note that SDu and SDv only perform computations on plaintext, and server is only in charge of system initialization. Therefore, their computation cost is negligible compared to the other entities.

Since the fog computing platform in current stage possesses the resource comparable to that of a smart phone, we have implemented the Paillier Cryptosystem on an Android mobile phone. The model number of the phone is Huawei Honor 3C (H30-U10) with the system parameters as: ARM Cortex-A7 4-core CPU @1.3GHz, 2GB memory and 4.2.2 Android version. When $|n|$ = 1024, the average running time (1000 iterations) for the exponentiation in $\mathbb{Z}_{n^{2}}$ is 55.493 milliseconds, the time for the multiplication in $\mathbb{Z}_{n^{2}}$ is 0.201 milliseconds and the time for the multiplication in $\mathbb{Z}_{n}$ is 0.101 milliseconds. It is obvious that the cost of multiplication is negligible compared to that of exponentiation. The computational cost for different entities is as shown in Table 1.

Another notable thing is that the evaluation here implicitly assumes the IoT environment devices are as powerful as the smart phone. This is true for some IoT applications which use mobile phones or vehicles to upload environmental information. However, for the applications utilizing low power sensors as EDs, the Paillier operations are still too heavy. To circumvent this issue, the sensor may transmit its data to nearby more powerful device for conducting the crypto-operations. For example, the wristband could connect with the mobile phone for processing and uploading data.

Communication Overhead: In this part, the communication overhead during SVD computation is evaluated. Note that for Paillier Cryptosystem, the ciphertext space is $\mathbb{Z}_{n^{2}}$. Thus, the bit length of one ciphertext is $2|n|$. The overhead of each communication flow is as shown in Table 2.

For the ”ED$\rightarrow$ FD” communication flow, each ED transits total $N_{C}$ ciphertexts to its corresponding FD, so the communication overhead is $N_{C}\cdot 2|n|$. For the ”FDs $\rightarrow$ SDd” communication flow, the overall ciphertexts SDd gathers from all the FDs is $N\cdot N_{C}$, so the communication overhead is $N\cdot N_{C}\cdot 2|n|$. For the ”SDd $\rightarrow$ SDu” communication flow, SDd sends $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$ to SDu. Since the bit length of each entry in $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$ is $(2\kappa_{1}+2\kappa_{3}+\kappa_{N})$ and there are total $l^{2}$ entries, the communication overhead is $l^{2}(2\kappa_{1}+2\kappa_{3}+\kappa_{N})$. Similarly, the communication overhead for the ”SDd $\rightarrow$ SDv” communication flow is $N^{2}(2\kappa_{1}+2\kappa_{3}+\kappa_{l})$.

7 Applications

In this section, we describe the potential IoT applications which could utilize the proposed framework. Basically, the proposed framework could be applied if the application possesses the following characteristics: 1) the application collects the environmental information for data analysis; 2) the data analysis is based on SVD; 3) the number of data analysis tasks is huge; 4) the environmental information is considered as privacy by the application users. Actually, the last two characteristics are the motivation of this work. The large amount of data analysis tasks motivates us to analyze data on fog computing platform. The privacy concern requires the analysis being privacy-preserving. Moreover, since different applications may utilize the result of SVD operation in different ways, we discuss how to adopt the proposed framework to achieve the purposes of different applications. In the following, three applications with different objectives are given as example to show that we only need to add several additional procedures or make some slight adjustments, the framework could be adapted to specific applications, which means the proposed framework is flexible.

7.1 Abnormal Detection

Since SVD could find the singular values of data matrix which reflect the features of data, it has proposed to apply SVD or the closely related technique–principal component analysis into abnormal detection [4][5]. The basic idea is quite straightforward: since the abnormal data would cause higher variation on the first eigenvector than the normal data, the variation degree of the first eigenvector could be utilized as the indicator of abnormal. When new data comes in, the system could perform SVD on the data matrix and check how much the direction of the first eigenvector changes. Note that to get the correct first eigenvector, it is usually required to normalize the data before performing SVD. However, the currently proposed framework is the general form which does not contain the normalization procedure. Thus, in the below, we show how to adjust the framework to accomplish that goal. Note that the eigenvector used for detection is from matrix $\mathbb{U}$, thus we only need to make adjustments on the procedures which are related to computing $\mathbb{U}$.

To include the normalization procedure, the $W$ and $S$ need to be chosen such that $W>2{N^{3}d^{2}}$ and $S>2{N^{3}(d^{2}+2tWd+t^{2}W^{2})}$. Then, it needs to add some extra steps in the pre-computation operation of SDd and derandomization operation of SDu. The extra steps are shown as follows.

• •

Step-1. For each randomized data $d_{ik}^{{}^{\prime}}$, SDd performs

[TABLE]

Then SDd performs matrix multiplication as before and sends the resulting $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$ to SDu.

• •

Step-2. For each entry $e$ of $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$, SDu computes $e^{{}^{\prime}}=e\bmod S$ first. If $e^{{}^{\prime}}>N^{3}(d^{2}+2tWd+t^{2}W^{2})$, SDu further computes $e^{{}^{\prime}}=e^{{}^{\prime}}-S$. Then SDu computes $e^{{}^{\prime\prime}}=e^{{}^{\prime}}\bmod W$. If $e^{{}^{\prime\prime}}>{N^{3}d^{2}}$, SDu further computes $e^{{}^{\prime\prime}}=e^{{}^{\prime\prime}}-W$. At last, SDu computes $e^{{}^{\prime\prime}}=\frac{e^{{}^{\prime\prime}}}{N^{2}}$. We denote the resulting matrix as $\mathbb{A}^{*}\cdot(\mathbb{A}^{*})^{T}$.

• •

Step-3. Let $e^{*}_{ij}$ denote the $i$th row and $j$th column entry of $\mathbb{A}^{*}\cdot(\mathbb{A}^{*})^{T}$. SDu computes the standard deviation of the $i$th dimensional data as

[TABLE]

Then for each entry $e^{*}_{ij}$, SDu computes $e^{*}_{ij}=\frac{e^{*}_{ij}}{\delta_{i}\delta_{j}}$. Finally, the resulting matrix is the correlation matrix generated from normalized data. The SDu could perform eigenvalue decomposition on the matrix as defined in the framework to find the first eigenvector and perform detection. One additional advantage of the proposed framework is that when new data arrives or the server wants to delete certain data, SDd only needs to add or substract the related randomized data, which is very convenient.

The correctness of normalization. The $d_{ik}^{*}$ is implicitly formed by

[TABLE]

Let $\hat{d_{ik}}$ denote $(Nd_{ik}-\sum_{m=1}^{N}d_{im})$, $\hat{z_{ik}}$ denote $(Nz_{ik}-\sum_{m=1}^{N}z_{im})$ and $\hat{r_{ik}}$ denote $(Nr_{ik}-\sum_{m=1}^{N}r_{im})$. We have $-Nd<\hat{d_{ik}}<Nd$, $-Nt<\hat{z_{ik}}<Nt$ and $-Nt<\hat{r_{ik}}<Nt$. The $(i,j)$th entry of $\mathbb{A}^{{}^{\prime}}\cdot(\mathbb{A}^{{}^{\prime}})^{T}$ is implicitly formed by

[TABLE]

It is easy to infer that $-N^{3}d^{2}<\sum_{k=1}^{N}\hat{d_{ik}}\cdot\hat{d_{jk}}<N^{3}d^{2}$ and $-N^{3}(d^{2}+2tWd+t^{2}W^{2})<\sum_{k=1}^{N}\hat{d_{ik}}\cdot\hat{d_{jk}}+\sum_{k=1}^{N}[(\hat{z_{ik}}\cdot\hat{d_{jk}}+\hat{z_{jk}}\cdot\hat{d_{ik}})\cdot W+\hat{z_{ik}}\hat{z_{jk}}W^{2}]<N^{3}(d^{2}+2tWd+t^{2}W^{2})$. On the other hand, we have $S>2{N^{3}(d^{2}+2tWd+t^{2}W^{2})}$ and $W>2{N^{3}d^{2}}$. Thus, if $e_{ij}^{{}^{\prime}}$, which is $(e_{ij}\bmod S)$, is larger than $N^{3}(d^{2}+2tWd+t^{2}W^{2})$, it means $\sum_{k=1}^{N}\hat{d_{ik}}\cdot\hat{d_{jk}}+\sum_{k=1}^{N}[(\hat{z_{ik}}\cdot\hat{d_{jk}}+\hat{z_{jk}}\cdot\hat{d_{ik}})\cdot W+\hat{z_{ik}}\hat{z_{jk}}W^{2}]<0$ and SDu needs to substract $S$ from the $e_{ij}^{{}^{\prime}}$ to get the correct result. Similarly, when $e_{ij}^{{}^{\prime\prime}}$, which is $(e_{ij}^{{}^{\prime}}\bmod W)$, is larger than $N^{3}d^{2}$, it means $\sum_{k=1}^{N}\hat{d_{ik}}\cdot\hat{d_{jk}}<0$, SDu needs to substract $W$. Therefore, during Step-2, SDu could recover the value of $e_{ij}^{{}^{\prime\prime}}=\sum_{k=1}^{N}\hat{d_{ik}}\cdot\hat{d_{jk}}$. It is obvious that $e_{ij}^{*}=\frac{e_{ij}^{{}^{\prime\prime}}}{N^{2}}=\sum_{k=1}^{N}(d_{ik}-\frac{1}{N}\sum_{m=1}^{N}d_{im})(d_{jk}-\frac{1}{N}\sum_{m=1}^{N}d_{jm})$ and $\sqrt{\frac{e_{ii}^{*}}{N-1}}=\sqrt{\frac{\sum_{k=1}^{N}(d_{ik}-\frac{1}{N}\sum_{m=1}^{N}d_{im})^{2}}{N-1}}=\delta_{i}$. The last step of dividing standard deviation is also straightforward.

7.2 Localized Recommendation System

SVD is also the underlying matrix factorization algorithm of Latent Semantic Indexing (LSI) which is widely used in information retrieval. Therefore, SVD is capable of capturing the latent relationships among customers and products. In this part, we describe how to utilize the proposed framework to build up a recommendation system.

Imagine that there are tens of restaurants in the local region where you live. You have already been some of them and want to try a new one, let’s say restaurant $p$, tonight. Before you go there, you would like to get a reputation score about $p$ from other people in the same region. If the score is too low, you may change the plan. We call the recommendation of local resource as localized recommendation. One may wonder why we consider a localized food recommendation system when there are already some centralized food review sites. The reason is that the localized food recommendation system has three advantages over the centralized food review sites. In specific, the first advantage is that the localized food recommendation system allows much more small restaurants to be included in the system while the centralized food review sites normally only have the data of the restaurants of middle or larger size. For example, the canteens in a university campus or even the food windows inside a canteen could be recorded in the localized food recommendation system while it is hardly possible for the centralized food review sites to maintain the information of the vast amount of tiny food stalls. The second advantage is that the rating vectors uploaded by the reviewers in the localized food recommendation system could often provide meaningful recommendations to the query user because those people live in the same local region and they are likely to have been in the same restaurants. Thus, the size of data matrix required in localized recommendation system is much smaller. The third advantage is that the localized food recommendation system could handle the service variance better. To be specific, the rating in the centralized food review sites often just reflects a single experience of the reviewer. However, the service of the restaurant may change over time, which reduces the information value contained in the rating. For the localized food recommendation system, the ratings keep being updated since people revisit the local restaurants frequently. Considering this effect, if a user has not been a local restaurant for a relatively long time, he would also require an updated reputation score from the localized recommendation system.

Since the personal taste is considered as privacy by many people and the recommendation tasks could appear in different regions frequently and concurrently, to get the local reputation score, we should utilize the proposed framework to conduct the SVD-based collaborative filtering as described in [6]. The detailed procedures are as follows:

• •

*Step-1. *The user $c$ uploads his rating vector to FD with his mobile phone and informs FD that he is interested in restaurant $p$. FD collects the rating vectors from other users inside this region. Note that to remove sparsity, each user fills in the ratings of unknown restaurants with his average rating.

• •

Step-2. FD uploads randomized data to SDd. SDd, SDu and SDv conduct some extra steps for normalizing the data matrix and perform SVD to get $\mathbb{U}$, $\mathbb{V}$ and $\Sigma$. The extra normalization steps of SDd and SDu are the same as the extra steps introduced in the above ”Abnormal Detection” application. Note that different with the ”Abnormal Detection” application, the recommendation application requires SDd to send the $(\mathbb{A}^{{}^{\prime}})^{T}\cdot\mathbb{A}^{{}^{\prime}}$ to SDv as well. Since the extra normalization steps needed by SDv could be straightforward derived from the steps of SDu, we omit the detailed description of the extra steps of SDv here.

• •

Step-3. SDu and SDv reduce $\mathbb{U}$, $\mathbb{V}$ and $\Sigma$ to $k$ dimension, and compute \mathbb{U}_{k}$$\Sigma_{k}^{\frac{1}{2}} and \Sigma_{k}^{\frac{1}{2}}$$\mathbb{V}_{k}^{T} respectively, i.e. SDu holds \mathbb{U}_{k}$$\Sigma_{k}^{\frac{1}{2}} and SDv holds \Sigma_{k}^{\frac{1}{2}}$$\mathbb{V}_{k}^{T}. Let \mathbb{U}_{k}$$\Sigma_{k}^{\frac{1}{2}}(c) denote the row of \mathbb{U}_{k}$$\Sigma_{k}^{\frac{1}{2}} which contains the information of user $c$ and \Sigma_{k}^{\frac{1}{2}}$$\mathbb{V}_{k}^{T}(p) denote the column of \Sigma_{k}^{\frac{1}{2}}$$\mathbb{V}_{k}^{T} which contains the information of restaurant $p$. The reputation score of restaurant $p$ for user $c$ is computed as \mathbb{U}_{k}$$\Sigma_{k}^{\frac{1}{2}}(c)\Sigma_{k}^{\frac{1}{2}}$$\mathbb{V}_{k}^{T}(p). Note that we use z-scores for normalization, so we do not need to add the user average back as in [6].

• •

Step-4. SDu and SDv send \mathbb{U}_{k}$$\Sigma_{k}^{\frac{1}{2}}(c) and \Sigma_{k}^{\frac{1}{2}}$$\mathbb{V}_{k}^{T}(p) to SDd for reputation score computation. To prevent SDd from inferring information, SDu and SDv randomize \mathbb{U}_{k}$$\Sigma_{k}^{\frac{1}{2}}(c) and \Sigma_{k}^{\frac{1}{2}}$$\mathbb{V}_{k}^{T}(p) with $W$ and $S$ respectively. For example, let $u_{i}$ denote the $i$th entry of \mathbb{U}_{k}$$\Sigma_{k}^{\frac{1}{2}}(c), SDu randomizes it as $u_{i}+z_{i}W+r_{i}S$.

• •

Step-5. SDd multiplies the two randomized vectors and sends the result ${score_{p}}^{{}^{\prime}}$ to FD. Since FD knows $W$ and $S$, it could recover the reputation score as $score_{p}={score_{p}}^{{}^{\prime}}\bmod S\bmod W$ and send $score_{p}$ to user $c$.

From the above description, it could be shown that the proposed framework could be utilized to compose a recommendation system without any difficulties. Also, note that in the above example, the server does not participate in the process, which means the proposed framework completes all the workload in the edge tier.

7.3 Data Compression

Since SVD could provide the best lower rank approximations of original data matrix, it is largely applied for compressing data. The data compression of SVD is quite straightforward: the matrix $\Sigma$ chooses $k$ largest diagonal values and sets the rest as 0. $\mathbb{U}$ and $\mathbb{V}$ retain the corresponding $k$ columns accordingly, then the matrix $\mathbb{U}_{k}\Sigma_{k}\mathbb{V}_{k}^{T}$ is the best rank $k$ approximations of original matrix. In the proposed framework, SDu and SDv could simply upload the trunked $\mathbb{U}_{k}$, $\Sigma_{k}$ and $\mathbb{V}_{k}$ to server to achieve this purpose.

Besides the lower rank approximation, the eigenvectors from $\mathbb{U}$ could be utilized to perform dimension reduction which is also a kind of data compression. In the framework, SDu could randomize the eigenvectors corresponding to the first several largest singular values. Then SDd could perform scalar product between the randomized user data and the received randomized eigenvectors. After the results of scalar product are recovered by SDu, it could upload the dimension-reduced data to server.

8 Related Works

In literature, there are a few works which are related to privacy-preserving SVD computation. Polat et al. [21] proposed a SVD-based collaborative filtering scheme in which the data privacy is protected by randomized perturbation. However, their scheme has been proven unsecure by [22]. Note that the randomization in this work does not have the feature of the randomized perturbation in [21]. Thus, the technique in [22] is infeasible for our work. Canny et al. [23] proposed a collaborative filtering scheme which achieves the SVD computation with privacy-preserving. However, their scheme is specifically designed for the recommendation application. Han et al. [24] proposed a secure protocol for SVD computation. However, their scheme could only support the computation between two parties. Hegedűs et al. [25] proposed a private SVD computation for low rank approximation in distributed P2P systems. Compared to our work, the works in [23, 24, 25] have limited applications and require considerable iterations for convergence which brings heavy overhead. Duan et al. [26] proposed a privacy-preserving framework which supports the computation of the learning algorithms which could be expressed as iterative form. Their work could support many learning algorithms while also requiring multiple rounds for the convergence of algorithms, which brings considerable overhead. For example, their scheme needs 83 minutes to compute the SVD for the Enron Email Data set which is a $150\times 150$ matrix while our work would need $N_{C}=$ 15 ciphertexts to aggregate the 150-dimensional data for each of the 150 users and only takes 499 seconds in total. Note that the evaluation in [26] sums up the computation time for all users even the computation of each user is actually performed concurrently. For fair comparison, our evaluation also accumulates the computation time of all users. Thus, the overall time cost is $4\times 55.493\times 150\times 15=499437$ milliseconds.

9 Conclusions

In this paper, a flexible fog computing framework for privacy-preserving SVD computation has been proposed. The framework divides the SVD calculation into two eigenvector decomposition operations and distributes the two tasks to different fog devices. The security analysis shows that the user data privacy is preserved during transmission, aggregation and eigenvector decomposition. The possible attacks from the second layer fog devices are also analyzed and the resistance of the framework against the potential attacks is discussed. The performance analysis has indicated the capacity of the framework and shows that the data dimension is the most important factor influencing the efficiency of the system. Moreover, three IoT applications are given as examples to demonstrate that the proposed framework is flexible enough to adapt to different applications. Compared with the existing works, our framework could support large scope of applications with relatively small resource consumption.