A Federated Filtering Framework for Internet of Medical Things

TL;DR

This paper introduces a federated filtering framework for IoMT devices that reduces communication costs and preserves privacy by predicting data at a fog server using shared models, with theoretical error bounds and practical validation.

Contribution

It proposes a novel federated filtering approach with theoretical analysis of error bounds and demonstrates significant communication savings in healthcare IoMT applications.

Findings

Saves up to 95% of communication cost

Maintains data privacy and low latency

Validates effectiveness with real healthcare data

Abstract

Based on the dominant paradigm, all the wearable IoT devices used in the healthcare sector also known as the internet of medical things (IoMT) are resource constrained in power and computational capabilities. The IoMT devices are continuously pushing their readings to the remote cloud servers for real-time data analytics, that causes faster drainage of the device battery. Moreover, other demerits of continuous centralizing of data include exposed privacy and high latency. This paper presents a novel Federated Filtering Framework for IoMT devices which is based on the prediction of data at the central fog server using shared models provided by the local IoMT devices. The fog server performs model averaging to predict the aggregated data matrix and also computes filter parameters for local IoMT devices. Two significant theoretical contributions of this paper are the global tolerable perturbation error (${To{l_F}}$) and the local filtering parameter ($\delta$); where the former controls the decision-making accuracy due to eigenvalue perturbation and the later balances the tradeoff between the communication overhead and perturbation error of the aggregated data matrix (predicted matrix) at the fog server. Experimental evaluation based on real healthcare data demonstrates that the proposed scheme saves upto 95\% of the communication cost while maintaining reasonable data privacy and low latency.

Tables

Table 1. TABLE I: The description of main symbols.

Symbol	Description
$N_i$	$i^{th}$ IoMT device
$Y$	Global matrix
$Y_i\left(t\right)$	The $i^{th}$ column of the global matrix
$\hat{\cdot }$	Perturbed version of the original symbol
${\delta }_i$	The $i^{th}$ filter parameter
${\theta }_i$	Prediction model of $i^{th}$ IoMT data
$e$	Mean square error function
$\alpha$	Learning rate/step size
$\lambda$	Eigen value of a matrix
$\Delta$	The perturbation error

Equations

$$e_i(t)=\frac{1}{2}\sum _{i=1}^n{\left({\hat{Y}}_i(t)-Y_i(t)\right)}^2$$

$${\hat{Y}}_i(t)={\theta }_i^T{Y}_i(t)$$

$${\theta }_i(t)={\theta }_i(t-1)+{\alpha }_i(t)\cdot e_i(t)\cdot Y_i(t)$$

$$0\le {\alpha }_i(t)\le \frac{1}{P_Y}$$

$$\begin{array}{r}\Vert \Delta \Vert =\Vert Y^{T}W+W^{T}Y+W^{T}W\Vert \\ \le \Vert Y^{T}W\Vert +\Vert W^{T}Y\Vert +\Vert W^{T}W\Vert \end{array}$$

$$\begin{array}{r}E({\Vert \Delta \Vert }_F)\le 2E\left({\Vert Y^{T}W\Vert }_F\right)+E\left({\Vert W^{T}W\Vert }_F\right)\\ \le 2\sqrt{E\left({\Vert Y^{T}W\Vert }_F^2\right)}+\sqrt{E\left({\Vert W^{T}W\Vert }_F^2\right)}\end{array}$$

$$E\left(\sqrt{\frac{1}{n}\sum _{i=1}^n{\left({\hat{\lambda }}_i-{\lambda }_i\right)}^2}\right)\le E\left(\frac{{\Vert \Delta \Vert }_F}{n}\right)\le To{l}_F$$

$$\begin{array}{r}E({\Vert \Delta \Vert }_F)\le 2\sqrt{\frac{1}{m^{2}n}Tr\left(Y^{T}Y\right)\cdot \sum _{i=1}^n{\sigma }_i^2}\\ +\sqrt{\left(\frac{1}{m}+\frac{1}{n}\right)\cdot \sum _{i=1}^n{\sigma }_i^4}\end{array}$$

$$E({\Vert \Delta \Vert }_F)\le To{l}_F$$

$$\begin{array}{r}To{l}_F=2\sqrt{\frac{1}{m^{2}n}Tr\left(Y^{T}Y\right)\cdot \sum _{i=1}^n{\sigma }_i^2}\\ +\sqrt{\left(\frac{1}{m}+\frac{1}{n}\right)\cdot \sum _{i=1}^n{\sigma }_i^4}\end{array}$$

$$\begin{array}{r}\delta =\frac{\sqrt{\frac{3Tr(Y^{T}Y)}{m}+3\cdot To{l}_F\cdot \sqrt{nm+m^2}}-\sqrt{\frac{3\cdot Tr(Y^{T}Y)}{m}}}{\sqrt{m+n}}\end{array}$$

$${\eta }_i(t)=\sum _{i=1}^K\frac{n_k}{n}{\theta }_i(t-1)$$

$$\hat{Y}(t)={\eta }_i^{T}Y(t)$$

$$⟨To{l}_F⟩=To{l}_F/\sqrt{\frac{\sum {\lambda }_i^2}{n}}$$

$$\eta =\sum _n\frac{d_n}{E_n\cdot r_n\cdot TTI}$$

Full text

I Introduction

World Health Organization (WHO) recently reports [1] a global health workforce shortage of 12.9 million during the coming decade. This expected shortage accompanied by various other factors have inspired a slow but steady paradigm shift from conventional healthcare to the smart healthcare [2, 3]. The smart healthcare enables patients with round the clock monitoring and feedback and is also expected to automate critical operations inside ICU [4]. Internet of Things (IoT) is widely accepted [5] as a crucial driver to the connected healthcare paradigm. Allied Market Research predicts [6] a global market capital for IoT healthcare to reach 136.8 billion US dollar by 2021, moreover today we already have 3.7 million connected internet of medical things (IoMT) devices.

A typical wearable IoMT device consists of a tiny battery which in most cases is nonchargeable [7], and this leads to disposal of the equipment once it is out of charge. A significant cause of speedier disposition of IoMT devices is due to the dominant cloud computing paradigm [8] of pushing all the collected data to the distant cloud servers for analytics and decision making. This phenomenon incurs a significant loss of power due to high communication overhead. Moreover, it also exposes the aggregated sensitive medical data to the security risks. This paper considers the problem of high power loss, exposure medical data privacy and high latency in cloud based healthcare analytics. It is an interesting problem as it has social implications also governments [9] and industries (Cisco [10], Microsoft [11]) are investing a lot of money and resources to develop a future healthcare infrastructure.

This paper presents an algorithmic framework namely Federated Filtering Framework (FFF) (Fig. 1) for IoMT supported by theoretical analysis. The proposed framework presents an alternate solution to the issues of energy efficiency, latency and privacy for resource-constrained IoMT devices. In brief, each IoMT device computes a local model of the data and shares this model with the fog server. The fog server’s role is threefold. First, it predicts a data matrix (aggregated data matrix) using aggregated model average (Section V); second, it computes and delivers filter parameters for all the IoMT devices and finally performs decision making using the aggregated data matrix. To control the eigenvalue perturbation of the data matrix that may compromise the decision accuracy this paper derives a theoretical relation between the local filtering parameter with the global tolerable eigenvalue perturbation using Matrix Perturbation Theory (MPT).

Overall, the contributions of the paper are as follows: (i) a theoretical relationship between local time series filtering and perturbation error of aggregated data matrix (ii) the implementation of federated decision making framework using filters, (iii) a lightweight fully unsupervised local subroutine (algorithm 1), (iv) the filter model averaging (algorithm 2) that preserves the privacy and demands few updates, (v) a practical framework for IoMT data aggregation.

The article is organised in the following fashion. Section II discusses the related work. Section III presents the system model. Section IV presents the theoretical analysis. Section V presents the kernel of the paper which is Federated Filtering Framework. Section IV shows the experimental evaluation, and finally, the article concludes by highlighting the significant contributions and future work.

II Related Work

This section compares the proposed framework with three closely related genres of research that includes IoT in healthcare, prediction based IoT systems and federated learning approaches in networks.

II-A IoT in Healthcare

The dominant paradigm for IoT based healthcare analytic systems [5] can be categorized as cloud computing-based health monitoring and mobile computing based health monitoring. Both the scenarios mentioned above very frequently push data to the server (cloud server/mobile device) for decision making. This paper is firmly against the continuous transmission of data and presents a prediction based data aggregation scheme with error bounds to ensure the fidelity of the decision making. Some recent use cases of IoT based healthcare analytics such as [12, 13] also advocates centralized decision making, however, both of them lacks a theoretical formulation to ensure decision-making accuracy.

II-B Prediction based IoT systems

The literature [14] reports several prediction based approaches for reducing the communication overhead in sensor networks. The prediction [15] based approaches are categorized into single prediction approaches and dual prediction approaches. In the case of single prediction approaches the system performs prediction in only one location whereas in the case of dual prediction approaches the system performs prediction at a local node along with the central server. Some notable prediction schemes applicable for both the categories mentioned above are adaptive filtering scheme [16], Autoregressive filter, Autoregressive Integrated Moving Average filter (ARIMA) [15], Kalman filtering and machine learning techniques [17]. Although some of the prior approaches can provide better accuracy for the model generation at the IoT device however given the severe computational constraints of the IoMT devices these approaches are not practical for local processing. Moreover, none of the earlier approaches shows any relationship between local and global processing using theoretical upper bounds.

II-C Federated Learning in Networks

The effectiveness of federated averaging algorithm for distributed training proposed by Mcmahan et al. [18] provides strong motivation to develop a federated filtering framework for IoMT devices. Moreover, there are also other notable distributed optimization approaches [19, 20] that improves communication efficiency. All the distributed and federated approaches in the literature are highly complex to run in a tiny IoMT device furthermore, they aim to perform decision making at the user device. The proposed Federated Filtering Framework on the other hand proposes a very lightweight subroutine for the local IoMT device and also aims to perform decision making at the server using local shared model.

III System Model

The system model considers a massive IoMT scenario where $n$ number of IoMT devices are cumulatively working towards sensing a particular phenomenon. All the IoMT devices are connected to the fog server(s) using Wi-Fi links. Each IoMT device $\left\{{{N_{1}},\dots,{N_{n}}}\right\}\in{N_{i}}$ generates a time series data stream. This paper assumes a centrally aggregated matrix $Y$ also known as global matrix (real) of size $m\times n$ where each column (${Y_{i}}$ ) represents a particular IoMT device, and each row has a sensor reading of every 30 seconds. This generation of a global matrix $Y$ requires continuous transmission of data to the fog server. However, this paper doesn’t advocate a continuous push and therefore proposes a prediction based framework. The fog server generates an aggregated data matrix ($\hat{Y}$); i.e. a predicted data matrix with perturbation and as earlier ${\hat{Y}_{i}}$ represents a column vector of the data matrix. The perturbation in the global data matrix is due to errors caused by filtering and predicton. The formation of aggregated data matrix is discussed in Section V. The fog server’s role is threefold. Firstly it estimates/predicts the perturbed data matrix ($\hat{Y}$), and secondly it computes and delivers filter parameter (${\delta_{i}}$) for all the IoMT devices, and finally, it performs decision making using the perturbed data matrix. Table 1 shows some important notations.

In the beginning, all the IoMT nodes train the prediction model by running several instances of Least Mean Square (LMS) filter (section IV A). Both the local IoMT device and the fog server uses the same prediction scheme. The local IoMT device runs a local processing subroutine as described in Algorithm 1 and the fog server runs Algorithm 2.

IV Theoretical Analysis

IV-A Adaptive Filtering at IoMT Devices

Adaptive filters are typically implemented for signals with non-stationary statistics and where no prior information is available. A typical adaptive filter is depicted in Fig. 2. Among various adaptive filters [21] this paper selects Least Mean Square (LMS) filter [22] for local processing inside the IoMT node, since it has a very low computational overhead.

Let for an IoMT device ${N_{i}}$ at time t the predicted IoMT sensor vector ${\hat{Y}_{i}}(t)$ be a linear approximation of the real sensor vector ${Y_{i}}(t)$. The LMS adaptive filter embedded inside the IoMT devices aims to minimise error the function $e(t)$, which is the least mean square approximation between the predicted sensor vector and the real sensor vector.

[TABLE]

The relationship between the predicted sensor vector ${\hat{Y}_{i}}(t)$ (output of LMS filter) and the real sensor vector ${Y_{i}}(t)$ is as follows.

[TABLE]

The LMS filter relies on the stochastic gradient descent (SGD) optimisation, this approach takes iterative steps (${\alpha_{i}}$) towards the steepest decrease of the error function ${e_{i}}(t)$. Eq. 3 shows the LMS update rule also known as Widrow-Hoff learning rule.

[TABLE]

Based on the empirical observation [21] to ensure convergency the step size ${\alpha_{i}}(t)$ should satisfy the following.

[TABLE]

where ${P_{Y}}=\frac{1}{M}\sum\limits_{j=1}^{M}{{{\left|{{Y_{i}}(j)}\right|}^{2}}}$, and M is the number of iterations taken for training the LMS filter.

IV-B Perturbation Analysis at Fog Server

The filter parameters play a key role in balancing the tradeoff between the desirable loss of decision accuracy (by allowing perturbation to $\hat{Y}$) and low communication overhead. This paper uses the matrix perturbation theory [23] to bound the perturbation error ($\Delta$) of the perturbed data matrix which in turn affects the decision accuracy. The fog server generates a perturbed data matrix $\hat{Y}=Y+W$, where $W$ is the perturbation/filtering error and column elements of $W$, ${W_{i}}\in\left[{-{\delta_{i}},{\delta_{i}}}\right]$. Let the ${\lambda_{i}}$ and ${\hat{\lambda}_{i}}$ denote the eigenvalues of the real covariance matrix $A=\frac{1}{m}{Y^{T}}Y$ and the perturbed covariance matrix $\hat{A}=\frac{1}{m}{\hat{Y}^{T}}\hat{Y}$ respectively.

The norm of the perturbation error matrix $\Delta=A-\hat{A}$ can be formulated using the property of triangle inequality [24] is depicted as follows.

[TABLE]

The goal here is to determine an upper bound for the expectation of RHS in the above inequality.

This paper assumes that all the column vectors of $W$ are independent and all the column elements are i.i.d random variables with zero mean ($\mu=0$) and variance $\sigma_{i}^{2}\approx\sigma_{i}^{2}({\delta_{i}})$ along with fourth moment as $\mu_{i}^{4}=\mu_{i}^{4}({\delta_{i}})$.

Using Jensen inequality $E(x)\leq\sqrt{E({x^{2}})}$.

[TABLE]

Based on Mirsky’s theorem [23].

[TABLE]

[TABLE]

[TABLE]

The Eq. 9 presents an upper bound ($To{l_{F}}$) on the perturbation error caused due to local filtering at IoMT devices and estimation of perturbed data matrix using outdated shared model.

[TABLE]

Similar upper bounds can also be derived using spectral norm ${\left\|.\right\|_{2}}$, moreover this paper selects Frobenious norm ${\left\|.\right\|_{F}}$ for no particular reason.

IV-C Uniform filter parameter selection

This paper assumes an independent and uniform distribution of IoT filter parameter within the interval $\left[{-{\delta_{i}},{\delta_{i}}}\right]$. Moreover we also assume a homogeneous filter parameter allocation among all the IoMT devices, therefore ${\delta_{i}}=\delta$ and ${\sigma_{i}}=\frac{{{\delta^{2}}}}{3}$. Solving Eq. 10 for $\delta$.

[TABLE]

The Eq. 11 provides a relationship between local filtering and the global perturbation error, that plays a crucial role in balancing the tradeoff between local filtering at IoMT devices and the global eigen perturbation error.

V Federated Filtering Framework

Federated Filtering Framework (FFF) since the system is based on a loose federation of the participating devices (IoMT devices) those are coordinated by the central fog server. The FFF consists of two crucial protocols first, the local data processing protocol and the second is global data processing and coordination protocol.

V-A Local Processing Protocol at IoT Device

Given the severe resource constraints in computation for IoMT devices, this paper proposes a very lightweight filtering protocol for local processing. The local filtering is based on LMS adaptive filter (section). The IoMT devices computes a local prediction model ${\theta_{i}}$ (Eq. 10) from the collected data and share this model with the fog server. Now assuming ${\theta_{i}}$ as the current prediction model and ${\delta_{i}}$ as the latest filtering parameter for the ${N_{i}}$ IoMT device. ${N_{i}}$ at any time t tracks the deviation of predicted sensor vector ${\hat{Y}_{i}}(t)$ from real sensor vector ${Y_{i}}(t)$ using ${W_{i}}(t)={Y_{i}}(t)-{\hat{Y}_{i}}(t)$. Whenever $\left|{{W_{i}}(t)}\right|>{\delta_{i}}$ the IoMT device updates the prediction model ${\theta_{i}}(t)$ and resets ${W_{i}}\left(t\right)$ to zero. The updated prediction model along with a small amount of sample data is shared with the fog server. However the LMS filter incurs negligible computational overhead that enables the IoMT device to run multiple instances of filtering for better accuracy. The above mentioned details for local processing at IoMT devices is summarized in algorithm 1.

V-B Federated Processing at Fog Server

At the beginning of each round the fog servers updates the current prediction models with the newly shared models. The fog server selects a random fraction K of the n participating IoMT devices. This paper selects a random fraction of IoMT devices [18] since the decision accuracy degrades beyond a certain number. The step size ${\alpha_{i}}(t)$ is kept constant based on the empirical result (section IV). The fog server aggregates the model using Eq. 12.

[TABLE]

Thereafter the fog server predicts the perturbed data matrix using the following equation.

[TABLE]

The perturbed data matrix $\hat{Y}(t)$ is used for decision making. The impact of eigen perturbation error on the decision accuracy can be studied in [24]. The fog server continuously tracks $E({\left\|\Delta\right\|_{F}})>To{l_{F}}$ . Once the data matrix perturbation error exceeds the tolerable perturbation error threshold, the fog shares an updated filter parameter and summons all IoMT devices to share their updated prediction model. The above mentioned scheme is summarized as Algorithm 2.

Advantages: The proposed framework minimises the communication overhead (section VI) by limiting the number of transmissions to the central server. The algorithm 2, i.e. the model averaging makes it practically impossible to extract an individual model from the average model; that ensures privacy to sensitive medical data. Furthermore, the fog server, unlike a cloud server, is located closer to the source, which reduces the latency.

VI Performance Evaluation

In this section, we present some experimental results based on real IoT health data. The results include the prediction using the filter model averaging by the fog server, the plot of communication overhead while varying local filtering parameter and the overall scalability of the proposal concerning energy efficiency. The experiments are performed using a publicly available111http://archive.ics.uci.edu/ml/datasets/MHEALTH+Dataset real IoT health dataset known as MHEALTH (mobile health) data. The dataset comprises body motion and vital signs recordings for ten volunteers of diverse profile while performing 12 physical activities. For our experiments, we have only considered the chest accelerometer sensor reading, i.e. columns 1-3 and the right lower arm gyro sensor time series data, i.e. column 18-20.

Based on section IV/C we assume a homogeneous filter parameter for all the IoMT devices. We initially distribute the data equally among 50 IoMT devices and compute normalized tolerable perturbation error as shown in Eq. 14.

[TABLE]

We present the relationship between the normalized tolerable perturbation error and the uniform filter parameter in Fig. 3. Moreover Fig. 3 depicts a roughly linear relationship between the normalized tolerable perturbation error and the local filter parameter. It is also intuitive since whenever one increases the $\left\langle{To{l_{F}}}\right\rangle$ , the filter at IoMT devices passes more data.

Next, we present the prediction performance of the filter model averaging scheme (Algorithm 2) by the fog server. Due to space limitations, we offer prediction results of two different IoMT devices (Fig. 4). As discussed in section both the local and the global filtering uses the same LMS filter. The available sophisticated techniques that provide better accuracy cannot be used at the fog server since those techniques must also be feasible for local processing at IoMT devices. Given the severe resource constraints in power and computation, the sophisticated methods cannot be used by IoMT devices for local processing.

Towards this end, we plot the communication overhead as a function of filter parameter. We observe that in Fig. 5 the communication cost can be massively reduced even with a tolerable perturbation error. We have achieved upto 95% reduction in transmissions with various tolerable perturbation error. This supports our claim that the proposed framework can provide a good tradeoff between communication efficiency and eigen perturbation error of data matrix.

Finally, we examine the scalability [25] of the proposed scheme for small as well as a large number of devices. The energy efficiency ($\eta$ ) of the system with n number of IoMT devices [26] can be computed as:

[TABLE]

Where ${d_{n}}$ is the total volume of data to be uploaded, ${E_{n}}$ is the average energy consumed to deliver a single packet, ${r_{n}}$ is total number of data packets to be uploaded by all the IoMT devices and $TTI=1$ is transmission time interval which is constant to all packets. It is evident from the plot (Fig. 6) that our FFF scheme is highly scalable compared to other recent researches such as AM-DR [16] and well known ARIMA [15]. Based on the plot, the energy efficiency increases with the number of devices. Therefore the proposed framework can also be extended to a massive IoMT scenario.

VII Conclusion

This paper considers open challenges concerning energy efficiency, privacy and latency for smart healthcare analytics. This paper derives a theoretical upper bound on the eigenvalue perturbation and further formulates a relationship between the local quantization at IoMT devices with the global perturbation error at fog server. Based on the theoretical infrastructure this paper proposes two subroutines first for the local filtering at the IoMT device and the second for the central fog server. The proposed framework cuts down 95% of the communication overhead. Moreover, the use of perturbed data matrix (predicted data) instead of using real global matrix for decision making ensures better privacy and the low proximity of fog server provides low latency. Future work includes formulating a general relation between decision accuracy and perturbation error and developing an IoMT testbed for verifying the proposed framework.

VIII Acknowledgement

This work is jointly supported by the National Natural Science Foundation of China under Grant 61871062 and Grant 61771082, also the Program for Innovation Team Building at Institutions of Higher Education in Chongqing under Grant CXTDX201601020. The Authors want to extend their gratitude to Prof. Pingping Zhang of Chongqing University of Posts and Telecommunications, Chongqing, China for her help with the mathematical formulation. Sunny Sanyal is the corresponding author.