Distributed mining of time--faded heavy hitters

TL;DR

This paper introduces P2PTFHH, a distributed algorithm for mining time-faded heavy hitters in P2P networks, combining gossip protocols with sketch data structures to achieve high accuracy and scalability.

Contribution

It is the first distributed method for time-faded heavy hitter mining in unstructured P2P networks, integrating gossip protocols with augmented sketch data structures.

Findings

Retains high accuracy and error bounds of FDCMSS

Proves convergence and correctness of the distributed algorithm

Demonstrates near real-time processing capability on large datasets

Abstract

We present \textsc{P2PTFHH} (Peer--to--Peer Time--Faded Heavy Hitters) which, to the best of our knowledge, is the first distributed algorithm for mining time--faded heavy hitters on unstructured P2P networks. \textsc{P2PTFHH} is based on the \textsc{FDCMSS} (Forward Decay Count--Min Space-Saving) sequential algorithm, and efficiently exploits an averaging gossip protocol, by merging in each interaction the involved peers' underlying data structures. We formally prove the convergence and correctness properties of our distributed algorithm and show that it is fast and simple to implement. Extensive experimental results confirm that \textsc{P2PTFHH} retains the extreme accuracy and error bound provided by \textsc{FDCMSS} whilst showing excellent scalability. Our contributions are three-fold: (i) we prove that the averaging gossip protocol can be used jointly with our augmented sketch data structure for mining time--faded heavy hitters; (ii) we prove the error bounds on frequency estimation; (iii) we experimentally prove that \textsc{P2PTFHH} is extremely accurate and fast, allowing near real time processing of large datasets.

Tables

Table 1. Table 1: Experiment values

$𝝆$	$\varphi$	$𝒑(\times {\mathrm{𝟏𝟎}}^{\mathrm{𝟑}})$	$𝒔𝒘(\times {\mathrm{𝟏𝟎}}^{\mathrm{𝟑}})$	$𝒓$	$𝒇𝒐$
{0.9, 1.1, 1.3, 1.5}	{0.01, 0.02, 0.03, 0.04}	{1, 5, 10, 15}	{1.5, 2.5, 3.5, 4.5}	{21, 24, 27}	{1, 2, 3}
1,2	0.02	5	2.5	24	1

Equations

$$\left|\mathcal{S}\right|=||\text{f}|{|}_1,$$

$${\hat{f}}_v-f_v\le {\hat{f}}^{min}\le \lfloor \frac{||\text{f}|{|}_1}{k}\rfloor ,v\in \mathcal{U}.$$

$${\sigma }_r^2=\frac{1}{p-1}\sum _{l=1}^p{\left(w_{r,l}-\bar{w}\right)}^2,$$

$$\mathbb{E}[{\sigma }_{r+1}^2]\approx \mathbb{E}[2^{-\psi }]\mathbb{E}[{\sigma }_r^2].$$

$$\frac{\mathbb{E}[{\sigma }_{r+1}^2]}{\mathbb{E}[{\sigma }_r^2]}=\mathbb{E}[2^{-\psi }];$$

$$\left|w_{r,l}-\bar{w}\right|<\sqrt{(p-1){\sigma }_0^2}\overline{ϵ}<p\bar{w}\overline{ϵ}$$

$$\mathbb{E}[{\sigma }_r^2]=\mathbb{E}[2^{-\psi }{]}^r{\sigma }_0^2;$$

$$\mathbb{P}[{\sigma }_r^2\ge \frac{\mathbb{E}[{\sigma }_r^2]}{{\delta }_g}]\le {\delta }_g;$$

$$\mathbb{P}[{\sigma }_r^2<\frac{\mathbb{E}[{\sigma }_r^2]}{{\delta }_g}]\ge 1-{\delta }_g.$$

$$\mathbb{P}[\sum _{l=1}^p{\left(w_{r,l}-\bar{w}\right)}^2<(p-1)\frac{{\gamma }^r{\sigma }_0^2}{{\delta }_g}]\ge 1-{\delta }_g.$$

$$ma{x}_{l\in [p]}{\left(w_{r,l}-\bar{w}\right)}^2\le \sum _{l=1}^p{\left(w_{r,l}-\bar{w}\right)}^2<(p-1)\frac{{\gamma }^r{\sigma }_0^2}{{\delta }_g},$$

$$ma{x}_{l\in [p]}\left|w_{r,l}-\bar{w}\right|<\sqrt{(p-1){\sigma }_0^2}\sqrt{\frac{{\gamma }^r}{{\delta }_g}}.$$

$$ma{x}_{l\in [p]}\left|w_{r,l}-\bar{w}\right|<\sqrt{(p-1){\sigma }_0^2}\sqrt{\frac{{\gamma }^r}{{\delta }_g}}<p\bar{w}\overline{ϵ}.$$

$$f_i\le {\hat{f}}_i\le f_i+\frac{eC}{2w};$$

$$I_{\mathcal{N}}(x):=\left\{{\begin{array}[]{*{20}c}{f_{\mathcal{N}}(x)}&{x\in N},\\ 0&{x\notin N},\\ \end{array}}\right.$$

$$\left|\mathcal{N}\right|:=Card(\mathcal{N})=\sum _{x\in N}{I}_{\mathcal{N}}(x),$$

$$\left|N\right|:=Card(N)=\sum _{x\in N}1.$$

$${\mathcal{D}}_r^{\prime }=({\mathcal{D}}_{r,1},{\mathcal{D}}_{r,2},\dots ,{\oslash }_2({\mathcal{D}}_{r,i}\oplus {\mathcal{D}}_{r,j}),\dots ,{\oslash }_2({\mathcal{D}}_{r,i}\oplus {\mathcal{D}}_{r,j}),\dots ,{\mathcal{D}}_{r,p}),$$

$${\mathcal{N}}_r^{\prime }=({\mathcal{N}}_{r,1},{\mathcal{N}}_{r,2},\dots ,{\oslash }_2({\mathcal{N}}_{r,i}\uplus {\mathcal{N}}_{r,j}),\dots ,{\oslash }_2({\mathcal{N}}_{r,i}\uplus {\mathcal{N}}_{r,j}),\dots ,{\mathcal{N}}_{r,p}).$$

$${\mathcal{D}}_{\infty }=\left({\mathcal{D}}_{\mathrm{avg}},{\mathcal{D}}_{\mathrm{avg}},\dots ,{\mathcal{D}}_{\mathrm{avg}}\right)$$

$${\mathcal{N}}_{\infty }=\left({\mathcal{N}}_{\mathrm{avg}},{\mathcal{N}}_{\mathrm{avg}},\dots ,{\mathcal{N}}_{\mathrm{avg}}\right),$$

$${\tilde{\mathbf{F}}}_r^{\prime }=({\tilde{\mathbf{f}}}_{r,1},{\tilde{\mathbf{f}}}_{r,2},\dots ,\frac{1}{2}({\tilde{\mathbf{f}}}_{r,i}+{\tilde{\mathbf{f}}}_{r,j}),\dots ,\frac{1}{2}({\tilde{\mathbf{f}}}_{r,i}+{\tilde{\mathbf{f}}}_{r,j}),\dots ,{\tilde{\mathbf{f}}}_{r,p}),$$

$${\tilde{\mathbf{F}}}_{\infty }=({\mathbf{f}}_{\mathrm{avg}},{\mathbf{f}}_{\mathrm{avg}},\dots ,{\mathbf{f}}_{\mathrm{avg}})$$

$${\mathbf{f}}_{\mathrm{avg}}=({\bar{f}}_1,{\bar{f}}_2,\dots ,{\bar{f}}_m),$$

$$\left|{\tilde{q}}_{r,l}-\frac{1}{p}\right|<\overline{ϵ}.$$

$$\frac{1}{p}-\overline{ϵ}<\frac{1}{{\tilde{p}}_{r,l}}<\frac{1}{p}+\overline{ϵ}.$$

$$\frac{1}{1+p\overline{ϵ}}<\frac{{\tilde{p}}_{r,l}}{p}<\frac{1}{1-p\overline{ϵ}}$$

$$\frac{p}{1+{ϵ}^{\ast }}<{\tilde{p}}_{r,l}<\frac{p}{1-{ϵ}^{\ast }}$$

$$\left|{\tilde{f}}_{r,l,i}-\frac{f_i}{p}\right|<f_i\overline{ϵ}.$$

$$\frac{f_i}{p}(1-{ϵ}^{\ast })<{\tilde{f}}_{r,l,i}<\frac{f_i}{p}(1+{ϵ}^{\ast })$$

Full text

1 Introduction

Distributed algorithms supporting large scale decentralized systems are being studied and developed in order to cope with complex infrastructures (peer-to-peer and sensor networks, IoT devices) and to face the fast paced growing of both the volume and dimension of data, coupled with the increasing information flow deriving by novel applications in disparate fields such as data mining, machine learning, artificial intelligence, optimization and control, and social networking.

Truly distributed solutions (i.e., we exclude here the class of centralized algorithms in which an agent acts as a coordinator) can be shown to provide the highly desirable properties of resilience and scalability, by empowering each individual agent (also known as peer) with information sensing and decision making. However, exploiting the communication network underlying the agents to design a protocol in which the peers interact and collaborate towards a common goal can be quite challenging.

One way to cope with this issue is to impose a well-known structured topology (e.g., a spanning-tree), whose properties can then be leveraged by the distributed protocol. A different approach, which shall be used in this paper, is the use of a gossip–based protocol [20]. In particular, we are interested in distributed averaging, which is a consensus algorithm serving as a foundational tool for the information dissemination of our distributed algorithm.

The task of distributed averaging is to drive the states of the nodes of a network towards an agreed common value with regard to the initial values held by the agents; this value is the average of the agents’ initial values. In such protocol, the agents share their status information only with a few other connected agents that are called neighbors.

A synchronous gossip–based protocol consists of a sequence of rounds in which each peer randomly selects one or more peers, exchanges its local state information with the selected peers and updates its local state by using the received information. Therefore, gossip can be thought as a simple way of self-organizing the peer interactions in consensus seeking.

In this paper, we present our P2PTFHH (Peer–to–Peer Time–Faded Heavy Hitters) distributed algorithm. Building on distributed averaging, we design a gossip–based version of our sequential FDCMSS (Forward Decay Count-Min Space-Saving) algorithm for distributed mining of time–faded heavy hitters on unstructured P2P networks [5]. To the best of our knowledge, this is the first distributed protocol designed specifically for this task. The problem of mining heavy hitters (also known as frequent items) is considered a fundamental data mining task and has been extensively studied. Among the many possible applications, we recall here its wide applicability in the context of sensor data mining, for business decision support, analysis of web query logs, network measurement, monitoring and traffic analysis. In particular, we are interested to the problem of distributed mining of time–faded heavy hitters, a model that puts heavier weights on recent batches of streaming data than older batches [12].

Our contributions are three-fold: (i) we formally prove that the averaging gossip protocol can be used jointly with our augmented sketch data structure for mining time–faded heavy hitters; (ii) we prove the error bounds on frequency estimation; (iii) we experimentally prove that P2PTFHH is extremely accurate and fast, allowing near real time processing of large datasets.

This paper is organized as follows. We recall in Section 2 relevant related work and in Section 3 preliminary definitions and concepts that shall be used in the rest of the manuscript. We present in Section 4 our P2PTFHH (Peer–to–Peer Time–Faded Heavy Hitters) algorithm and formally prove in Section 5 its correctness. Next, we provide extensive experimental results in Section 6. Finally, we draw our conclusions in Section 7.

2 Related Work

Misra and Gries [33] designed in 1982 the first algorithm for mining frequent items. However, their solution was not proposed as a streaming algorithm, even though it works in a streaming context. The Lossy Counting and Sticky Sampling algorithms by Manku et al. [31], published in 2002, were considered the first streaming algorithms until 2003, the year in which the Misra and Gries algorithm was independently rediscovered and improved, with regard to its running time, by Demaine et al. (the so-called Frequent algorithm) [19] and Karp et al. [26]. The key idea to decrease the per item processing time was the use of a better data structure, based on a hash table. A few years later, Metwally et al. presented Space-Saving, a novel algorithm providing a significant improvement with regard to the output’s accuracy, whilst retaining the same running time and space required to process the input stream.

All of the previous algorithms for detecting heavy hitters are commonly known as counter–based, since they use a set of counters to keep track of the frequent items. Other algorithms are instead known as sketch–based, owing to their use of a sketch data structure to monitor the input stream. A sketch is usually a bi-dimensional array data structure containing a counter in each cell. Hash functions map the stream items to corresponding cells in the sketch. In this class of algorithms, we recall here CountSketch by Charikar et al. [10], Group Test [14] and Count-Min [13] by Cormode and Muthukrishnan and hCount [25] by Jin et al.

Parallel algorithms designed for fast mining of heavy hitters span both the message–passing and shared–memory paradigms. Cafaro et al. [7] [8] [3] designed parallel versions of the Frequent and Space-Saving algorithms for message–passing architectures. Regarding shared-memory, we recall here parallel versions of Lossy Counting and Frequent by Zhang et al. [42] [43], parallel versions of Space-Saving by Dat et al. [17], Roy et al. [36], and Cafaro et al [6]. Parallel algorithms for fundamental frequency–based aggregates, including heavy hitters were designed by Tangwongsan et al. [38]. Among the accelerator based algorithms, exploiting GPU (Graphics Processing Unit) and the Intel Phi, we find Govindaraju et al. [23], Erra and Frola [22] and Cafaro et al. [1] [6].

Mining Correlated Heavy Hitters (CHH) has been recently proposed by Lahiri et al. [28]; common applications requiring accurate mining of CHHs are related to network monitoring and management, and to anomaly and intrusion detection. For instance, taking into account the stream of pairs (source address, destination address) consisting of IP packets traversing a router, the data mining task is the identification of the nodes accounting for the majority of the traffic passing through that router (these are the frequent items over a single dimension); however, given a frequent source, it is desirable and important as well to simultaneously discover the identity of the destinations receiving the majority of connections by the same source. After mining the first dimension to detect the most important sources, the second dimension is mined to detect the frequent destinations in the context of each identified source, i.e., the stream’s correlated heavy hitters. Epicoco et al. [21] proposed a fast and more accurate algorithm for mining CHHs; a parallel, message-passing based version of this algorithm appears in Pulimeno et al. [35].

The above algorithms, when processing an input stream, do not discount the effect of old data. In practice, all of the items are given equal weight. However, this is not appropriate for those applications based on the underlying assumption that recent data is more useful and valuable than older, stale data. One way to handle such as a situation is using the so-called sliding window model [18] [34]. The key idea is the use of a temporal window to capture fresh, recent items. this window periodically slides forward, allowing detection of only those frequent items falling in the window.

The time–fading model [12] puts heavier weights on recent batches of streaming data than older batches. This is achieved by fading the frequency count of older items: each item in the stream has an associated timestamp that shall be used to determine its weight, by means of a decaying factor $0<\lambda<1$. Therefore, instead of an item’s frequency count, algorithms compute its decayed count (also known as decayed frequency) through a decay function assigning greater weight to more recent items. The older an item, the lower its decayed count is: in the case of exponential decay, the weight of an item occurred $n$ time units in the past, is $e^{-\lambda n}$, which is an exponentially decreasing quantity.

Noteworthy algorithms for mining time–faded heavy hitters include $\lambda$-HCount [11] by Chen and Mei, FSSQ (Filtered Space-Saving with Quasi–heap) [40] by Wu et al. and the FDCMSS algorithm [5] [2] by Cafaro et al. These algorithms are sketch–based variants of Count-Min. Whilst $\lambda$-HCount and FSSQ rely on a backward decay exponential function, FDCMSS provides support for both backward and forward decay functions [15]. For instance, by using a polynomial forward function, FDCMSS provides more flexibility with regard to time fading, since the time fades more slowly than by using an exponential function. Moreover, the use of forward decay allows FDCMSS dealing with out of order arrival of stream items easily. On the contrary, neither $\lambda$-HCount nor FSSQ can handle out of order items. Both $\lambda$-HCount and FSSQ require a dedicated data structure to keep track of frequent item candidates: $\lambda$-HCount uses a doubly linked list indexed by an hash function, whilst FSSQ uses a so–called Quasi-Heap, an heap in which heapify operations are delayed until necessary. Our FDCMSS algorithm does not require additional space beyond its data structure, a sketch in which each cell hosts a Space-Saving stream summary holding exactly two counters. We have shown that, fixing the same amount of space for all of the algorithms, FDCMSS outperforms $\lambda$-HCount and FSSQ with regard to the error committed. Moreover, FDCMSS provides the best error bound. Regarding the recall property (i.e., the ability of outputting all of the true frequent items), both $\lambda$-HCount and FSSQ guarantee 100% recall, whilst FDCMSS provides a probabilistic recall guarantee. Finally, with regard to the running time, FDCMSS is the fastest owing to its data structures and update strategy. A parallel message-passing based algorithm has been recently proposed in [4].

Many algorithms for distributed mining of heavy hitters have been designed, e.g., [9], [44], [27], [30], [39]. Among the gossip–based algorithms for mining heavy hitters on unstructured P2P networks, it is worth recalling here [37], [45], and [29].

The algorithms by Sacha and Montresor [37] and by Aem and Azkasap [45] are based on the same ideas. Indeed, the dataset to be mined is partitioned among the peers, which periodically exchange their local state consisting of the current subset of the whole dataset. The key assumption is that a peer is allowed to store the whole dataset (even though this requires obviously a huge amount of space), which is indeed obtained by means of the distributed averaging gossip interactions. In order to estimate $N$, the number of peers, the distributed averaging approach is used again as follows: one of the peers starts with a value equal to one and all of the others with a value equal to zero. It can be shown that the protocol converges to $1/N$, so that $N$ can be easily estimated. In particular, the convergence of the averaging gossip protocol has been investigated in [24]; here, the authors proved that the variance around the mean value being computed is reduced in each round by a specific convergence factor.

Exchanging the full local state, besides requiring a huge amount of space, also increases to communication complexity. Therefore, in [37] the authors suggest to exchange only the top-$k$ most frequent items where $k$ is a user’s defined parameter. The protocol’s termination condition requires for each peer that the subset consisting of the top-$k$ items does not change for a specified number of consecutive rounds. In [45], to cope with the communication complexity, the authors uses an additional data structure, which is an hash table storing all of the items (these items are never deleted, at the cost of increased space used) from which the algorithm randomly selects a specified number of items corresponding to a predefined message size. The termination condition requires two user’s defined parameters: $\epsilon$ and $convLimit$. The algorithm terminates when for each item, the absolute difference between the estimated frequency in the current and previous rounds is less than or equal to $\epsilon$ for at least $convLimit$ consecutive rounds.

Both [37] and [45] require space complexity linear in the length $n$ of the dataset; this allows solving the exact problem rather than the approximate problem, but at the expense of space.

In Lahiri and Tirthapura [29], the authors design an algorithm based on random sampling of the items and the averaging gossip protocol. Each item is assigned a random weight in the interval (0, 1) and a peer maintains and exchanges in each round the $t$ items whose weight is the lowest, with $t=\frac{128}{\psi^{2}}\ln\frac{3}{\delta}$; $\psi$ and $\delta$ denote respectively a threshold and a probability of failure. The proposed approach can only detect frequent items, since the algorithm does not provide frequency estimation. In particular, the output is a list of items that with high probability (defined by $\delta$) contains the frequent items (with regard to the $\psi$ threshold). Regarding the space used, for each of the $t$ items the algorithm stores a tuple consisting of four fields: the peer identifier, the item index in the peer’s local dataset, the item value and its random weight.

None of the previous gossip-based protocols has been designed to mine time–faded heavy hitters on unstructured P2P networks. To the best of our knowledge, P2PTFHH is the first distributed protocol designed specifically for this task.

3 Preliminary definitions

In this Section we introduce preliminary definitions and the notation used throughout the paper. We deal with an input data stream $\sigma$ consisting of a sequence of $n$ items drawn from a universe $\mathcal{U}$; without loss of generality, let $m$ be the number of distinct items in $\sigma$ i.e., let $\mathcal{U}=\{1,2,\ldots,m\}$, which we shall also denote by $[m]$. Let $f_{i}$ be the frequency of the item $i\in\mathcal{U}$ (i.e., its number of occurrences in $\sigma$), and denote the frequency vector by $\textbf{f}=(f_{1},\ldots,f_{m})$. Moreover, let $0<\phi<1$ be a support threshold, $0<\epsilon<1$ a tolerance such that $\epsilon<\phi$ and denote the 1-norm of f (which represents the total number of occurrences of all of the stream items) by $||\textbf{f}||_{1}$.

In this paper, we are concerned with the problem of detecting heavy hitters in a stream which is distributed among $p$ peers with the additional constraint that recent items are more relevant with regard to older items. In the time–faded model, recent items are weighted more than former items while in the sliding window model only the items’ occurrences which appear in a window sliding over time are considered. Our algorithm, P2PTFHH, works in the former model.

The time–fading model [30] [12] [11] [40] uses a fading frequency count of older items to represent the freshness of recent items. This is achieved by computing the item’s decayed frequency through the use of a decay function that assign greater weight to more recent occurrences of an item than to older ones: the older an occurrences is, the lower its decayed weight.

Definition 1**.**

Let $w(t_{i},t)$ be a decayed function which computes the decayed weight at time $t$ for an occurrence of the item $i$ arrived at time $t_{i}$. A decayed function must satisfy the following properties:

$w(t_{i},t)=1$* when $t_{i}=t$ and $0\leq w(t_{i},t)\leq 1$ for all $t>t_{i}$;* 2. 2.

$w$* is a monotone non-increasing function as time $t$ increases, i.e., $t^{\prime}\geq t\implies w(t_{i},t^{\prime})\leq w(t_{i},t)$.*

Related work has mostly exploited backward decay functions, in which the weight of an item is a function of its age, $a$, where the age at time $t>t_{i}$ is simply $a=t-t_{i}$. In this case, $w(t_{i},t)$ is given by $w(t_{i},t)=\frac{h(t-t_{i})}{h(t-t)}=\frac{h(t-t_{i})}{h(0)}$, where $h$ is a positive monotone non-increasing function.

The term backward decay stems from the aim of measuring from the current time back to the item’s timestamp. Prior algorithms and applications have been using backward exponential decay functions such as $h(a)=e^{-\lambda a}$, with $\lambda>0$ being the decaying factor.

In our algorithm, we use instead a forward decay function, defined as follows (see [15] for a detailed description of the forward decay approach). Under forward decay, the weight of an item is computed on the amount of time between the arrival of an item and a fixed point $L$, called the landmark time, which, by convention, is some time earlier than the timestamps of all of the items. The idea is to look forward in time from the landmark to see an item, instead of looking backward from the current time.

Definition 2**.**

Given a positive monotone non-decreasing function $g$, and a landmark time $L$, the forward decayed weight of an item $i$ with arrival time $t_{i}>L$ measured at time $t\geq t_{i}$ is given by $w(t_{i},t)=\frac{g(t_{i}-L)}{g(t-L)}$.

The denominator is used to normalize the decayed weight so that $w(t_{i},t)$ is always less than or equal to 1 as requested by Definition 1.

Definition 3**.**

The decayed frequency of an item $v$ in the input stream $\sigma$, computed at time $t$, is given by the sum of the decayed weights of all the occurrences of $v$ in $\sigma$: $f_{v}(t)=\sum_{v_{i}=v}w(t_{i},t)$.

Definition 4**.**

The decayed count at time $t$, $C(t)$, of a stream $\sigma$ of $n$ items is the sum of the decayed weights of all the items occurring in the stream: $C(t)=\sum_{i=1}^{n}w(t_{i},t)$.

The Approximate Time–Faded Heavy Hitters (ATFHH) problem is formally stated as follows.

Problem 1**.**

Approximate Time–Faded Heavy Hitters. Given a stream $\sigma$ of items with an associated timestamp, a threshold $0<\phi<1$ and a tolerance $0<\epsilon<1$ such that $\epsilon<\phi$, and letting $g$ be a decaying function used to determine the decayed frequencies and $t$ be the query time, return the set of items $H$, so that:

$H$* contains all of the items $v$ with decayed frequency at time $t$ $f_{v}(t)>\phi C(t)$ (decayed frequent items);*

• 2.

$H$* does not contain any item $v$ such that $f_{v}(t)\leq(\phi-\epsilon)C(t)$.*

In the following, when clear from the context, the query time shall be considered an implicit parameter, and we shall write $f_{v}$ and $C$ in place of $f_{v}(t)$ and $C(t)$. Our algorithm uses an averaging gossip protocol for unstructured P2P networks in which the peers exchange their internal state consisting of a Count–Min sketch data structure augmented by a Space-Saving summary associated to each sketch cell. In the following, we recall the main properties of the Count–Min and Space-Saving algorithms for ordinary, non decaying frequencies and the averaging gossip protocol.

Count–Min is based on a sketch of $d\times w$ cells, where each item is mapped to by means of $d$ different pairwise independent hash functions. Count–Min solves the frequency estimation problem for arbitrary items with an estimation error $\epsilon\leq e/w$ and with probability $1-e^{-d}$. The algorithm may also be extended to solve the approximate frequent items problem as well, by using an additional heap data structure which is updated each time a cell is updated. Since in Count–Min the frequencies stored in the cells overestimate the true frequencies, a point query for an arbitrary item simply inspects all of the $d$ cells in which the query item is mapped to by the corresponding hash functions and returns the minimum of those $d$ counters.

Space-Saving is a counter-based algorithm solving the heavy hitters problem. It is based on a stream summary data structure consisting of a given number of counters $k\ll n$, $n$ being the length of the stream. Each counter monitors an item in the stream and tracks its frequency. A substitution strategy is used when the algorithm processes an item not already monitored and all of the counters are occupied.

Let $\sigma$ be the input stream and denote by $\mathcal{S}$ the summary data structure of $k$ counters used by the Space-Saving algorithm. Moreover, denote by $\left|\mathcal{S}\right|$ the sum of the counters in $\mathcal{S}$, by $f_{v}$ the exact frequency of an item $v$ and by $\hat{f}_{v}$ its estimated frequency, let $\hat{f}^{min}$ be the minimum frequency in $\mathcal{S}$. If there exist at least one counter not monitoring any item, $\hat{f}^{min}$ is zero.

Finally, denote by $\textbf{f}=(f_{1},\ldots,f_{m})$ the frequency vector. The following relations hold (as shown in [32]):

[TABLE]

[TABLE]

The gossip–based protocol [20] is a synchronous distributed algorithm consisting of periodic rounds. In each of the rounds, a peer (or agent) randomly selects one or more of its neighbors, exchanges its local state with them and finally updates its local state. The information is disseminated through the network by using one of the following possible communication methods: (i) push, (ii) pull or (iii) push–pull. The main difference between push and pull is that in the former a peer randomly selects the peers to whom it wants to send its local state, whilst in the latter it randomly selects the peers from whom to receive the local state. Finally, in the hybrid push–pull communication style, a peer randomly selects the peers to send to and from whom to receive the local state. In this synchronous distributed model it is assumed that updating the local state of a peer is done in constant time, i.e., with $O(1)$ worst-case time complexity; moreover, the duration of a round is such that each peer can complete a push–pull communication within the round.

We are interested in a specific gossip–based protocol, which is called distributed averaging, and can be considered as a consensus protocol. For the purpose of our theoretical analysis, we assume that peers and communication links do not fail, and that neither new peers can join the network nor existing peers can leave it (the so-called churning phenomenon). Therefore, the graph $G=(V,E)$ in which $V$ is the set of peers and $E$ models the connectivity, fully describing the underlying network topology, is not time-varying. However, it is worth noting here that our algorithm also works in time-varying graphs in which the network can change owing to failures or churning and we shall show an experimental evidence of that in Section 6.1, in which we discuss the effect of churn.

In uniform gossiping, a peer $i$ can communicate with a randomly selected peer $j$. Instead, in our scenario the communication among the peers is restricted to neighbor peers i.e., two peers $i$ and $j$ are allowed to communicate if and only if a one-hop communication link exists between them; we assume that communication links are bidirectional. Initially, each peer $i$ is provided with or computes a real number $v_{i}$; the distributed averaging problem requires designing a distributed algorithm allowing each peer computing the average $v_{{\rm avg}}=\frac{1}{p}\sum_{i=1}^{p}v_{i}$ by exchanging information only with its neighbors. Letting $v_{i}(r)$ be the peer $i$ estimated value of $v_{{\rm avg}}$ at round $r$, a gossip interaction between peers $i$ and $j$ updates both peers’ variables so that at round $r+1$ it holds that $v_{i}(r+1)=v_{j}(r+1)=\frac{1}{2}(v_{i}(r)+v_{j}(r))$. Of course, for a peer $i$ which is not gossiping at round $r$ it holds that $v_{i}(r+1)=v_{i}(r)$. It can be shown that distributed averaging converges exponentially fast to the target value $v_{{\rm avg}}$. In general, a peer is allowed to gossip with at most one peer at a time. In our algorithm, we allow each peer the possibility of gossiping with a predefined number of neighbors. We call fan-out, $fo$, of the peer $i$ the number of its neighbors with which it communicates in each round; therefore, $1\leq fo\leq\left|\{j:(i,j)\in E\}\right|$. Therefore, we explicitly allow two or more pairs of peers gossiping at the same time, with the constraint that the pairs have no peer in common. We formalize this notion in the following definition.

Definition 5**.**

Two gossip pairs of peers $(i,j)$ and $(x,y)$ are noninteracting if neither $i$ nor $j$ equals either $x$ or $y$.

In our algorithm multiple noninteracting pairs of allowable gossips may occur simultaneously. Non-interactivity is required in order to preserve and guarantee correctness of the results; in the literature non-interactivity is also called atomic push–pull communication: given two peers $i$ and $j$, if peer $i$ sends a push message to $j$, then peer $i$ can not receive in the same round any intervening push message from any other peer $k$ before receiving the pull message from $j$ corresponding to its initial push message.

It is worth noting here that our algorithm does not require explicitly assigning identifiers to the peers, and we do so only for convenience, in order to simplify the analysis; however, we do assume that each peer can distinguish its neighbors.

4 The algorithm

In this section, we start by recalling our sequential algorithm FDCMSS [5]. The key data structure is an augmented Count–Min sketch $\mathcal{D}$, whose dimensions $d$ (rows) and $w$ (columns) are derived by input parameters $\epsilon$, the error tolerance, and $\delta$, the probability of failure. Whilst every cell in an ordinary Count–Min sketch contains a counter used for frequency estimation, in our case a cell holds a Space-Saving summary with exactly two counters. The idea behind the augmented sketch is to monitor the time–faded items that the sketch hash functions map to the corresponding cells by an instance of Space-Saving with two counters, so that for a given cell we are able to determine a majority item candidate with regard to the sub-stream of items falling in that cell. In fact, we proved that, with high probability, if an item is frequent, then it appears as a majority item candidate in at least one of the $d$ sketch cells in which it falls.

Indeed, by using a Space–Saving summary with two counters in each cell, and letting $C_{i,j}$ denote the total decayed count of the items falling in the cell $\mathcal{D}[i][j]$, the majority item is, if it exists, the item whose decayed frequency is greater than $\frac{C_{i,j}}{2}$. The corresponding majority item candidate in the cell is the item monitored by the Space-Saving counter whose estimated decayed frequency is maximum.

The main idea of the distributed version of the FDCMSS algorithm is to let each peer process its local data stream with the FDCMSS algorithm. Then, the peers engage in a gossip–based distributed averaging protocol, exchanging their local state which consists of the augmented sketch data structure obtained after processing the input stream and an estimate of the number of peers in the network.

The algorithm’s initialization (pseudo-code provided as Algorithm 1) requires as input parameters $d$ and $w$, the dimensions of the local sketch; $\mathcal{N}_{l}$, the local dataset to be processed, being $l$ the peer’s identifier; $\epsilon$, the error tolerance; $\delta_{g}$, a probability of failure of the gossip protocol (i.e., related to the event that the gossip protocol fails to provide the theoretical properties that it should guarantee with high probability); $\phi$, the frequent support threshold; $p^{*}$, an estimate of the number $p$ of peers in the network (we only require $p^{*}\geq p$); $R$, the number of rounds to be performed by the distributed algorithm; and $\gamma$, the gossip protocol convergence factor, whose meaning shall be explained in the theoretical analysis section.

The initialization procedure starts calling the CreateSketch function which allocates and returns an augmented sketch: for each one of the $d\times w$ cells of the sketch, CreateSketch allocates a Space-Saving summary with two counters $c_{1}$ and $c_{2}$, both maintaining an item and a frequency value (given a counter $c_{j},j=1,2$, we denote by $c_{j}.i$ and $c_{j}.f$ respectively the counter’s item and its estimated decayed frequency). Moreover, CreateSketch selects $d$ pairwise independent hash functions $h_{1},\ldots,h_{d}:[m]\rightarrow[w]$, mapping $m$ distinct items into $w$ cells. It is worth noting here that each peer is required to select the same $d$ hash functions in order to guarantee the correctness of the results.

After the main data structures are created, each peer stores a local copy of the remaining input parameters. Then, the peer whose identifier is $l=1$ sets $\tilde{q}_{0,l}$ to 1 and all of the other peers sets this value to zero. The variable $\tilde{q}_{0,l}$ represents the estimate of peer $l$ at round [math] of the number $p$ of peers (to be obtained by using the distributed averaging protocol: indeed, upon convergence this value approaches $1/p$ with high probability).

Each peer processes its local data stream following the update procedure of the FDCMSS algorithm, shown in pseudo-code as Algorithm 2. For each item $i$ with timestamp $t_{i}$, the non normalized forward decayed weight $x$ of the item is computed, then the $d$ cells in which the item is mapped to by the corresponding hash functions $h_{j}(x),j=1,\dots,d$ are updated by using the Space-Saving item update procedure.

We shortly describe here the update procedure of the Space-Saving algorithm, SpaceSavingUpdate in Algorithm 2: let $\mathcal{S}$ denote the Space-Saving summary with two counters corresponding to the cell to be updated. When processing an item which is already monitored by a counter, its estimated frequency is incremented by the non normalized weight $x$. When processing an item which is not already monitored by one of the available counters, there are two possibilities. If a counter is available, it will be in charge of monitoring the item, and its estimated frequency is set to the non normalized weight $x$. Otherwise, if all of the counters are already occupied (their frequencies are different from zero), the counter storing the item with minimum frequency is incremented by the non normalized weight $x$. Then, the monitored item is evicted from the counter and replaced by the new item. This happens since an item which is not monitored can not have a frequency greater than the minimal frequency.

We represent the local state of a peer $l$ at round $r$ as a tuple $state_{r,l}$ consisting of the peer’s local sketch $\mathcal{D}_{r,l}$, and the estimate $\tilde{q}_{r,l}$.

In the P2PTFHH algorithm, all of the peers participate to the distributed gossip protocol for $R$ rounds (Gossip procedure in Algorithm 3). During each round, a peer increments $r$, the round counter, selects $fo$ (the fan-out) neighbours uniformly at random and sends to each of them its local state in a message of type push. Upon receiving a message, each peer executes the on_receive procedure, whose pseudo-code is shown as Algorithm 3. From the message, the peer extracts the message’s type, sender and state sent. A message is processed accordingly to its type as follows. A push message is handled in two steps. In the first one, the peer updates its local state by using the state received; this is done by invoking the update procedure that we shall describe later. In the second one, the peer sends back to the sender, in a message of type pull, its updated local state. A pull message is handled by a peer setting its local state equal to the state received.

The update procedure, which is the last procedure described by Algorithm 3, works as follows: the two local sketches $\mathcal{D}_{i}$ and $\mathcal{D}_{j}$ belonging respectively to the peers $i$ and $j$ are merged by invoking the merge procedure, producing the new sketch $\mathcal{D}$; since we want to implement a distributed averaging protocol, we scan the counters of each stream summary $\mathcal{S}$ in the cells of $\mathcal{D}$, and for each counter $c$ we update its frequency $c.f$ dividing it by 2; finally, we compute as required by the averaging protocol the estimate $\tilde{q}$ and return the updated state just computed.

Two sketches are merged by the merge procedure (pseudo-code provided as Algorithm 4) as follows: for every corresponding cell in the two sketches to be merged, the hosted Space–Saving summaries are merged following the steps described in [7], i.e., building a temporary summary $\mathcal{S}_{C}$ consisting of all of the items monitored by both $\mathcal{S}_{1}$ and $\mathcal{S}_{2}$. To each item in $\mathcal{S}_{C}$ is assigned a decayed frequency computed as follows: if an item is present in both $\mathcal{S}_{1}$ and $\mathcal{S}_{2}$, its frequency is the sum of its corresponding frequencies in each summary; if the item is present only in one of either $\mathcal{S}_{1}$ or $\mathcal{S}_{2}$, its frequency is incremented by the minimum frequency of the other summary. At last, in order to derive the merged summary, we take only the two items in $\mathcal{S}_{C}$ with the greatest frequencies and discard the others.

Once $R$ rounds of gossip have been executed, the user can invoke the query procedure (pseudo-code provided as Algorithm 5) on an arbitrary peer to retrieve the time–faded heavy hitters. This is done by computing $\tau$, a threshold that determines whether an item is a candidate heavy hitter or not, $\tilde{p}_{r,l}$, the estimate of $p$ and $\tilde{C}_{r,l}$, the global decayed count. Note that $\tau$ is defined in terms of $\epsilon^{*}$, whose meaning shall be explained in Section 5.4. The total decayed count can be computed by summing up the decayed frequencies stored by the Space–Saving counters inside each single cell in a row of the sketch data structure $\mathcal{D}$. It is worth noting here that the decayed frequencies are still non normalized; the normalization occurs dividing by $g(t-L)$, where $t$ is the query time and $L$ denotes a landmark time in the past. The query procedure initializes $H$, an empty set, and then it inspects each of the $d\times w$ cells in the sketch $\mathcal{D}$. For a given cell, we determine $c_{m}$, the counter in the data structure $\mathcal{S}$ with maximum decayed count. We compare the normalized decayed count stored in $c_{m}$ with the threshold $\tau=\phi\tilde{C}_{r,l}\frac{1-\epsilon^{*}}{1+\epsilon^{*}}$. If the normalized decayed frequency is greater, we pose a point query for the item $c_{m}.i$, shown in pseudo-code as the PointEstimate procedure. If the returned value is greater than the threshold $\tau$, then we insert in $H$ the pair $(c_{m}.i,p)$.

The point query for an item $j$ returns its estimated decayed frequency. After initializing the answer variable to infinity, we inspect each of the $d$ cells in which the item is mapped to by the corresponding hash functions, to determine the minimum decayed frequency of the item. In each cell, if the item is stored by one of the Space-Saving counters, we set answer to the minimum between answer and the corresponding counter’s decayed frequency. Otherwise (none of the two counters monitors the item $j$), we set answer to the minimum between answer and the minimum decayed frequency stored in the counters. Since the frequencies stored in all of the counters of the sketch are not normalized, we return the normalized frequency answer dividing by $g(t-L)$. At the end of the query procedure the set $H$ is returned.

5 Correctness

In this Section, we prove that in our algorithm every peer correctly converges to a suitable sketch data structure that can be queried to solve the Approximate Time–Faded Heavy Hitters problem. We also prove a bound on the frequency estimation error committed by our algorithm.

Before proceeding with our analysis, we need to recall the results by Jelasity et al. in [24], related to the convergence factor of the uniform gossiping protocol, and the main properties exhibited by the algorithm FDCMSS and by the procedure for merging FDCMSS sketches [4], that we use in Algorithm 4.

5.1 Jelasity’s averaging algorithm

In the work by Jelasity et al. [24], the proposed averaging protocol is interpreted as a distributed variance reduction algorithm, fully equivalent to a centralized algorithm which computes the average value among a set of elements. Consider a variance measure $\sigma_{r}^{2}$ defined as:

[TABLE]

where $w_{r,l}$ is the value held by the peer $l$ after $r$ rounds of the gossip algorithm and $\bar{w}=\frac{1}{p}\sum_{l=1}^{p}w_{0,l}$ is the mean of the initial values held by the peers. The authors in [24] state that if $\psi_{l}$ is a random variable denoting the number of times a node $l$ is chosen as a member of the pair of nodes exchanging their states during a round of the protocol, and each pair of nodes is uncorrelated, then the following theorem holds.

Theorem 1**.**

[24]*

If:*

the random variables $\psi_{1},\ldots,\psi_{p}$ are identically distributed (let $\psi$ denotes a random variable with this common distribution), 2. 2.

after a pair of nodes $(i,j)$ is selected, the number of times the nodes $i$ and $j$ shall be selected again have identical distributions,

then:

[TABLE]

The random variable $\psi$ only depends on how the pair of nodes are selected. From equation (4), the convergence factor is defined as:

[TABLE]

Therefore, the convergence factor depends on $\psi$ and, as a consequence, on the pair selection method. Jelasity et al. compute the convergence factor for different methods of pair selection; we are only interested in the one which allows simulating the distributed gossip–based averaging protocol. This method consists in drawing a random permutation of the nodes and then, for each node in that permutation, choosing another random node in order to form a pair. For this selection method, the convergence factor is $\gamma=\mathbb{E}[2^{-\psi}]=1/(2\sqrt{e})$.

We now derive from theorem 1 the following proposition.

Proposition 2**.**

Let $\delta_{g}$ be a user-defined probability, $w_{r,l}$ the value held by peer $l$ after $r$ rounds of the averaging protocol, $p$ the number of peers participating in the protocol, $\gamma=\mathbb{E}[2^{-\psi}]=1/(2\sqrt{e})$ the convergence factor, $\bar{w}$ the mean of the initial vector of values $\boldsymbol{w}_{0}$, i.e. $\bar{w}=1/p\sum_{l=1}^{p}w_{0,l}$ and $\bar{\epsilon}=\sqrt{\frac{\gamma^{r}}{\delta_{g}}}$ the error factor. Then, with probability greater than or equal to $1-\delta_{g}$ it holds that, for any peer $l$:

[TABLE]

Proof.

From equation (4) it follows that:

[TABLE]

where $\sigma_{0}^{2}$ depends on the distribution of the initial numbers among the peers. By the Markov inequality, it holds that:

[TABLE]

or

[TABLE]

Considering eqs. (3) and (7), it follows that:

[TABLE]

As a consequence, with probability at least $1-\delta_{g}$:

[TABLE]

which implies:

[TABLE]

The initial distribution $\sigma_{0}^{2}$ of the elements over the peers typically is not known in advance, but the worst case happens when only one peer has the whole quantity $p\bar{w}$ and the other $p-1$ peers hold the value [math]. In this case, it follows that $\sigma_{0}^{2}\leq p\bar{w}^{2}$, and hence, with probability $1-\delta_{g}$:

[TABLE]

This proves the proposition. ∎

Equation (6) gives an upper bound on the error made by any peer estimating the value $\bar{w}$ after $r$ rounds of the Jelasity’s averaging algorithm. This bound is probabilistic and valid with probability greater than or equal to $1-\delta_{g}$.

5.2 FDCMSS algorithm and merging of augmented sketches

We recall here the main results related to the theoretical properties of the FDCMSS algorithm. Cafaro et al. in [5] proved that FDCMSS solves the Approximate Time–Faded Heavy Hitters problem and give a bound of its frequency estimation error. In particular, the authors state that, with high probability, if a time–faded item is frequent, then, in at least one of the FDCMSS sketch cells where it is mapped to by the hash functions, it is a majority item with regard to the sub-stream of items falling in the same cell. Therefore, FDCMSS will detect it.

The following theorem holds, being $d\times w$ the dimensions of the sketch and $\phi$ the frequent threshold.

Theorem 3**.**

If an item $i$ is frequent, then it appears as a majority item candidate in at least one of the $d$ cells in which it falls, with probability greater than or equal to $1-(\frac{1}{2\phi w})^{d}$.

Moreover, regarding the error bound on frequency estimation of the FDCMSS algorithm, if $f_{i}$ is the exact decayed frequency of item $i$ in the stream $\sigma$ and $\hat{f}_{i}$ is the estimated decayed frequency of item $i$ returned by FDCMSS and $C$ is the total decayed count of all of the items in the input stream, then, the authors proved that the following error bound holds:

Theorem 4**.**

Denoting by $C$ the total decayed count, and by $d$ and $w$ the number of rows and columns of the augmented sketch data structure, then $\forall i\in[m]$, $\hat{f}_{i}$ estimates the exact decayed count $f_{i}$ of item $i$ at query time with error less than $eC/2w$ and probability greater than $1-e^{-d}$, i.e.:

[TABLE]

P2PTFHH follows the same structure of the gossip–based averaging protocol by Jelasity et al., where, instead of single values, two nodes communicating provide two sketches to be merged and averaged. The merge operation refers to the procedure introduced by Cafaro et al. in [4], where the authors proved that the merge procedure of two augmented sketch data structures preserves all of the properties of the sketch, including the fact that an FDCMSS sketch is 1-norm equivalent to a classical Count-Min sketch for the same input stream, i.e. the value of a cell in the Count-Min sketch is equal to the sum of the Space-Saving counters in the same cell of the FDCMSS sketch.

Therefore, if $\mathcal{D}_{1}$ and $\mathcal{D}_{2}$ are FDCMSS sketches which respectively refer to the streams $\mathcal{N}_{1}$ and $\mathcal{N}_{2}$, then the sketch $\mathcal{D}=\mathcal{D}_{1}\oplus\mathcal{D}_{2}$ is an FDCMSS sketch for the stream $\mathcal{N}=\mathcal{N}_{1}\uplus\mathcal{N}_{2}$, where we have used the symbol $\oplus$ to indicate the merge operation.

5.3 Convergence of P2PTFHH

In order to prove the convergence of P2PTFHH, we shall use multisets to represent the input streams and introduce some operations on multisets and sketches in order to replicate the average pattern of Jelasty et al. algorithm.

Definition 6**.**

A multiset $\mathcal{N}=(N,f_{\mathcal{N}})$ is a pair where $N$ is some set, called the underlying set of elements, and $f_{\mathcal{N}}:N\rightarrow\mathbb{N}$ is a function. The generalized indicator function of $\mathcal{N}$ is

[TABLE]

where the integer–valued function $f_{\mathcal{N}}$, for each $x\in N$, provides its frequency (or multiplicity), i.e., the number of occurrences of $x$ in $\mathcal{N}$. The cardinality of $\mathcal{N}$ is expressed by

[TABLE]

whilst the cardinality of the underlying set $N$ is

[TABLE]

A multiset, or bag, is defined by a proper set (the support set) and a multiplicity function: it is a set where elements can be repeated, i.e., an element in a multiset can have multiplicity greater than one.

For our specific application, we relax the above definition of multiset, allowing the multiplicity function $f_{N}$ to assume real values. Let $\mathcal{M}$ be the class of all of the multisets with support set included in $\mathcal{U}$. We introduce the operation $\oslash_{v}:\mathcal{M}\rightarrow\mathcal{M}$, so that $\oslash_{v}(\mathcal{N})=(N,f_{N}/v))$, i.e, the multiset $\oslash_{v}(\mathcal{N})$ has the same support set of $\mathcal{N}$, but each element has a fraction $1/v$ of the multiplicity it has in $\mathcal{N}$. When $v\in\mathbb{N}$, we have that $\biguplus_{i=1}^{v}\oslash_{v}(\mathcal{N})=\oslash_{\frac{1}{v}}(\oslash_{v}(\mathcal{N}))=\mathcal{N}$.

We can define a similar operation for an augmented FDCMSS sketch $\mathcal{D}$ and, by abusing the same symbol, say that $\oslash_{v}(\mathcal{D})=\mathcal{D^{\prime}}$, where $\mathcal{D^{\prime}}$ is obtained from $\mathcal{D}$ by dividing all of the frequencies’ values in $\mathcal{D}$ by $v$.

It is immediate to see that if $\mathcal{D}$ is an augmented sketch for $\mathcal{N}$, then $\oslash_{v}(\mathcal{D})$ is a augmented sketch for $\oslash_{v}(\mathcal{N})$ and theorems 3 and 4 continue to hold.

Algorithm 6, called AVG-Merge, is a simplified global version of the distributed P2PTFHH that simulates the distributed averaging protocol. AVG-Merge operates on the global state of the network by simulating the distributed P2PTFHH protocol and allowing us to simplify its theoretical analysis. GetPair, not reported here, is a procedure which selects a pair of nodes and meets the properties requested in theorem 1.

Initially, each peer computes a local augmented sketch on its input stream, through the execution of the FDCMSS algorithm, then the distributed protocol starts. The initial distributed state of the system can be represented by the vector of the local augmented sketches ${\boldsymbol{\mathcal{D}}_{0}=(\mathcal{D}_{0,1},\mathcal{D}_{0,2},\ldots,\mathcal{D}_{0,p})}$, where $p$ is the number of peers participating in the protocol. Another vector is naturally associated to $\boldsymbol{\mathcal{D}}_{0}$: the vector of the local input streams $\boldsymbol{\mathcal{N}}_{0}=(\mathcal{N}_{0,1},\mathcal{N}_{0,2},\ldots,\mathcal{N}_{0,p})$. We have that $\biguplus_{l=1}^{p}\mathcal{N}_{0,l}=\mathcal{N}$, where we denote by $\mathcal{N}$ the global input stream.

Each call to AVG-Merge corresponds to a round of P2PTFHH. It modifies $\boldsymbol{\mathcal{D}}_{r}$, the vector of the local sketches held by the peers at the end of round $r$, producing the vector $\boldsymbol{\mathcal{D}}_{r+1}$. Furthermore, implicitly also $\boldsymbol{\mathcal{N}}_{r}$, the vector of local input streams to which the sketches refer, changes to $\boldsymbol{\mathcal{N}}_{r+1}$. Indeed, let $\boldsymbol{\mathcal{D}}_{r}$ and $\boldsymbol{\mathcal{N}}_{r}$ be the vectors of the sketches owned by the peers and the corresponding partition of the input stream $\mathcal{N}$ after the $r$th round. Then, after each iteration of the main loop of AVG-Merge, letting $(i,j)$ be the pair of communicating peers, i.e. the pair selected by GetPair, the vector of sketches becomes:

[TABLE]

and the corresponding vector of partitions of the input stream shall change to:

[TABLE]

From what we said on the operations $\oplus$ and $\oslash$, after each elementary iteration of AVG-Merge, two invariants hold:

each peer $l$ owns a sketch $\mathcal{D}_{r,l}$ which is a correct augmented sketch data structure for the portion of input stream $\mathcal{N}_{r,l}$; 2. 2.

$\biguplus_{l=1}^{p}\mathcal{N}_{r,l}=\mathcal{N}$.

These invariants remain true after each iteration of the main loop of AVG-Merge and, consequently, after each call to AVG-Merge, that is after each round of the P2PTFHH distributed protocol, when we derive from the vectors $\boldsymbol{\mathcal{D}}_{r}$ and $\boldsymbol{\mathcal{N}}_{r}$, the new vectors $\boldsymbol{\mathcal{D}}_{r+1}$ and $\boldsymbol{\mathcal{N}}_{r+1}$.

We can state that, for $r\rightarrow\infty$, the two vectors $\boldsymbol{\mathcal{D}}_{r}$ and $\boldsymbol{\mathcal{N}}_{r}$ converge respectively to:

[TABLE]

and

[TABLE]

where $\mathcal{N}_{\rm avg}=\oslash_{p}(\mathcal{N})$ and $\mathcal{D}_{\rm avg}$ is a correct FDCMSS sketch for $\mathcal{N}_{\rm avg}$.

This means that all of the peers converge to a sketch of $\oslash_{p}(\mathcal{N})$, from which, for the properties of the operations $\oplus$ and $\oslash$, a correct sketch for $\mathcal{N}$ can be derived by computing $\oslash_{\frac{1}{p}}(\mathcal{D}_{\rm avg})$ (we actually need to know the number of peers, which is not always the case, but we shall see in the following how we can estimate $p$), i.e. P2PTFHH converges.

Thanks to the invariants discussed above, in order to prove the convergence of the local sketches to $\mathcal{D}_{\rm avg}$, it is enough to verify that the local input streams implicitly induced by the algorithm converge to $\mathcal{N}_{\rm avg}$.

We can represent each initial local input stream $\mathcal{N}_{0,l}$ for $l=1,2,\ldots,p$, as the frequencies’ vector of the items in that stream, $\boldsymbol{\tilde{f}}_{0,l}=(\tilde{f}_{0,l,1},\tilde{f}_{0,l,2},\ldots,\tilde{f}_{0,l,m})$. Each value $\tilde{f}_{0,l,i}$ corresponds to the frequency that item $i$ has in the initial local stream held by peer $l$. In this representation the operator $\oslash_{p}$ on a multiset translates to a multiplication of the frequencies’ vector corresponding to that multiset by the scalar $1/p$.

Now, the implicit transformation that the local streams of the selected peers, $i$ and $j$, undergo at each elementary iteration of AVG-Merge, i.e., equation (19), can be rewritten as:

[TABLE]

where $\boldsymbol{\tilde{F}}_{r}$ is a matrix whose columns are the peers’ vectors of frequencies after $r$ rounds, i.e. each $\boldsymbol{\tilde{f}}_{r,l}$ is the frequencies’ vector corresponding to the multiset ${\mathcal{N}}_{r,l}$. This matrix corresponds to the vector of multisets $\boldsymbol{\mathcal{N}}_{r}$ in equation (19).

Eventually, it can be recognized in equation (22) the elementary step of the Jelasity’s protocol applied componentwise to the frequencies’ vectors of peers $i$ and $j$. We already know that the Jelasity’s averaging protocol converges to the average of the values initially owned by the peers. Thus, for $r\rightarrow\infty$, $\boldsymbol{\tilde{F}}_{r}$ converges to:

[TABLE]

where $\boldsymbol{f}_{\rm avg}$ is:

[TABLE]

with $\bar{f}_{i}=\frac{1}{p}\sum_{l=1}^{p}\tilde{f}_{0,l,i},\text{ for }i=1,2,\ldots,m$ which is the representation as frequencies’ vector of the multiset $\mathcal{N}_{\rm avg}$ in equation (21), proving the convergence.

5.4 Gossip-based approximation

In our P2PTFHH algorithm, all of the peers participate to the averaging protocol by exchanging their internal state, i.e., by merging the local augmented sketches, $\mathcal{D}_{r,l}$, and averaging the local values $\tilde{q}_{r,l}$.

In order to estimate the number of peers, the averaging protocol is performed using the values initially distributed among the peers so that only one peer starts with a value equal to $1$ while the other $p-1$ peers start with values equal to [math]. Denoting with $\tilde{q}_{r,l}$ the value held by peer $l$ at round $r$, this quantity approaches the average value $1/p$ as the algorithm proceeds. We can bound the error for the estimation of the number of peers. Considering the equation 6, we have that :

[TABLE]

In fact, equation 25 can be easily derived from equation 6 by substituting $\bar{w}$ with the average value $1/p$. Since $\tilde{q}$ is an estimate of the inverse of the number of peer, we can write equation 25 introducing $\tilde{p}=1/\tilde{q}$:

[TABLE]

Since the number of peers is a positive number, we can set the constraint $\bar{\epsilon}<1/p$; under this constraint all of the members of the previous relation are positive, hence it holds that:

[TABLE]

The problem with equation (27) is that the estimation error bounds depend on $p$, but we may not know $p$ in advance. To overcome this problem, we introduce the value $p^{*}~{}\geq~{}p$, that is an estimate of the maximum number of peers we expect in the network, and we compute new bounds based on this value. Under the constraint $p^{*}\geq p$, we can be confident on the new computed bounds, though they may be weaker.

Let us set $\epsilon^{*}=p^{*}\bar{\epsilon}$. Given the constraint on $\bar{\epsilon}$, it holds that $0<\epsilon^{*}<1$, and, with probability $1-\delta_{g}$, for any peer $l=1,2,\ldots,p$:

[TABLE]

As discussed in Section 5.3, as the the P2PTFHH algorithm proceeds, the augmented Sketch $\mathcal{D}_{r,l}$, held by a peer $l$ after $r$ rounds approaches $\mathcal{D}_{\rm avg}$ which corresponds to the augmented sketch we would obtain had we applied FDCMSS to the stream $\mathcal{N}_{\rm avg}$, which is, in turn, the stream corresponding to the average frequency vector that we would obtain applying the Jelasty averaging protocol component wise to the local peers’ frequency vectors.

Denoting with $\tilde{f}_{r,l,i}$ the decayed frequency of the item $i$ in the stream $\mathcal{N}_{r,l}$, after $r$ rounds, and considering the equation 6, it holds that:

[TABLE]

Equation 29 can be easily derived from equation 6 by substituting $\bar{w}$ with the average value $f_{i}/p$. Considering the expression of $\epsilon^{*}$, it follows that, with probability $1-\delta_{g}$, for any peer $l=1,2,\ldots,p$:

[TABLE]

Finally, the total decayed count $C$ shall be estimated too. Let $\tilde{C}_{r,l}$ be the total decayed count estimate, we have that $\tilde{C}_{r,l}=\sum_{i=1}^{d}\tilde{f}_{r,l,i}$, thus, from the previous equation 30, it follows that, with probability $1-\delta_{g}$, for any peer $l=1,2,\ldots,p$:

[TABLE]

Moreover, it is worth noting here that, due to its properties, the augmented sketch data structure being 1-norm equivalent to the classical Count–Min sketch, each peer can determine the total decayed count estimate $\tilde{C}_{r,l}$, simply by summing up the values of all of the cells in one row of its sketch, $\mathcal{D}_{r,l}$.

5.5 Correctness and error bounds

We shall show here that given an augmented sketch data structure $\mathcal{D}_{r,l}$ obtained by the peer $l$ after $r$ rounds of the P2PTFHH algorithm, we can select a set of items and their corresponding estimated frequencies, thus solving the Approximate Time–Faded Heavy Hitters Problem. We shall also determine the error bounds on frequencies’ estimation and the relation among the number of the sketch cells to be used by each peer and the number $r$ of rounds to be executed in order to guarantee the false positives’ tolerance requested by the user within a given probability of failure.

Theorem 5**.**

*Given an input stream $\mathcal{N}$, with total decayed count $C$, distributed among $p$ peers, each one making use of an augmented sketch data structure of size $d\times w$, and given a threshold parameter $0<\phi<1$, after $r$ rounds of P2PTFHH, on any peer the QUERY procedure returns a set $H$ of items and their corresponding time–faded frequencies so that, with probability greater than $1-\delta_{n}$,

$H$* includes all of the items in $\mathcal{N}$ with frequency $f>\phi C$;

with $\delta_{n}=\delta_{g}+\left(\frac{1}{2\phi w}\frac{1+\epsilon^{*}}{1-\epsilon^{*}}\right)^{d}(1-\delta_{g})$, $\delta_{g}$ being the probability of failure pertaining to the gossip approximation.

Proof.

We recap the query procedure of the P2PTFHH algorithm: all of the sketch cells are inspected, if the majority item candidate in the current cell has a decayed frequency greater than the selection criteria $\tau=\phi\tilde{C}_{r,l}\frac{1-\epsilon^{*}}{1+\epsilon^{*}}$, then a PointEstimate query is issued to compute the estimated frequency $\hat{f}$ of the heavy hitter candidate and finally if $\hat{f}>\tau$ the item is inserted in the set $H$.

Hence, to be selected a heavy hitter must:

appear as a majority item candidate in at least one of the $d$ cells in which it is mapped to;

• 2.

have estimated frequency $\hat{f}$ greater than $\tau$.

We first prove that if an item is a heavy hitter (i.e. $f>\phi C$), then its estimated frequency is greater than $\tau$ with probability at least $1-\delta_{g}$. We recap the main relations we proved above, valid with probability at least $1-\delta_{g}$, for all of the items $i$ and any peer $l$ after $r$ rounds of the algorithm:

[TABLE]

Moreover, owing to the FDCMSS properties it holds that:

[TABLE]

From the relations (32)–(36), we can derive the following:

[TABLE]

Hence the following holds with probability at least $(1-\delta_{g})$:

[TABLE]

We now prove that a heavy hitter appears in at least one of the $d$ cells in which it is mapped to with probability at least $\left[1-\left(\frac{1}{2w\phi}\frac{1+\epsilon^{*}}{1-\epsilon^{*}}\right)^{d}\right](1-\delta_{g})$ and that the overall probability of P2PTMHH failing to retrieve all of the frequent items, or alternatively, the probability of P2PTFHH reporting false negatives, is $\delta_{n}$.

Let us take as a reference the state of a peer $l$ after the $r$th round of the algorithm and let $k$ be a frequent item for the global stream $\mathcal{N}$, i.e. $f_{k}>\phi C$. Let $\tilde{\mathcal{D}}_{r,l}[i][j]$, for $i\in\{1\ldots d\}$ and $j=h_{i}(k)$, be one of the $d$ cells in which the item $k$ is mapped to by the corresponding hash functions $h_{i}:[m]\rightarrow[w]$. Moreover, let $\tilde{f}^{(i,j)}_{min}$ be the minimum of the two counters available in the Space-Saving summary $\tilde{\mathcal{S}}_{i,j}$, which monitors the items falling in the cell $\tilde{\mathcal{D}}_{r,l}[i][j]$, and let $\hat{f}_{i,k}$ be the estimated decayed count of the item $k$ related to the local stream $\tilde{\mathcal{N}}_{r,l}$ and determined through the Space-Saving summary $\tilde{\mathcal{S}}_{i,j}$. The item $k$ shall not appear as a majority item candidate in any of the $d$ cells in which it is mapped to iff the following event happens:

[TABLE]

Let $\tilde{S}_{i,j}$ be the total decayed count of the items falling in the cell $\tilde{D}_{r,l}[i][j]$. By equation (2), for the minimum of the two counters in the cell $\tilde{D}_{r,l}[i][j]$, it holds that: $\tilde{f}^{(i,j)}_{min}\leq\frac{\tilde{S}_{i,j}}{2}$.

We must now determine the probability that the event described in equation (39) occurs for a given row $i$ of the sketch, i.e. $\Pr[\hat{f}_{i,k}\leq\tilde{f}^{(i,j)}_{min}]$, under the condition that $f_{k}>\phi C$.

Taking into account that $\tilde{f}^{(i,j)}_{min}\leq\frac{\tilde{S}_{i,j}}{2}$, it holds that:

[TABLE]

and, as proved in [5], $\mathbb{E}[\tilde{S}_{i,j}]=\frac{\tilde{C}_{r,l}}{w}$. Thus, using the Markov inequality we have:

[TABLE]

By applying equations (32)–(36) we have:

[TABLE]

Finally, taking into account that $k$ is a heavy hitter, i.e. $f_{k}>\phi C$, it holds that:

[TABLE]

We fail to identify a frequent item $k$ when that item is not reported as a majority item candidate in any of the $d$ cells $\tilde{D}_{r,l}[i][j]$ in which it falls. Thus, by equation (43), the probability of this event happening is:

[TABLE]

Considering that the equations (32)–(36) are correct with a given probability $(1-\delta_{g})$ (see Section 5.1), we can conclude that if an item is a heavy hitter, it is reported with probability greater than or equal to $\left[1-\left(\frac{1}{2\phi w}\frac{1+\epsilon^{*}}{1-\epsilon^{*}}\right)^{d}\right]\left(1-\delta_{g}\right)$, and that, consequently, the probability of failure of P2PTFHH retrieving all of the frequent items is:

[TABLE]

. ∎

Theorem 6**.**

*Given an input stream $\mathcal{N}$, with total decayed count $C$, distributed among $p$ peers, each one making use of an augmented sketch data structure of size $d\times w$, and given a threshold parameter $0<\phi<1$, after $r$ rounds of P2PTFHH, on any peer the QUERY procedure returns a set $H$ of items and their corresponding time–faded frequencies so that, with probability greater than or equal $1-\delta_{p}$,

$H$* does not include any item in $\mathcal{N}$ with frequency $f\leq(\phi-\epsilon)C$;

with a false positives tolerance $\epsilon=\frac{4\epsilon^{*}\phi}{(1+\epsilon^{*})^{2}}+\frac{e}{2w}\frac{1-\epsilon^{*}}{1+\epsilon^{*}}$, and a probability of failure $\delta_{p}=\delta_{g}+e^{-d}(1-\delta_{g})$, $\delta_{g}$ being the probability of failure pertaining to the gossip approximation.

Proof.

Owing to the FDCMSS properties, the estimated frequency of an item $i$ is bounded by the following relation, with probability at least $1-e^{-d}$:

[TABLE]

In order to compute the error, in terms of false positives’ tolerance, that we commit with the selection criterion $\hat{f}>\tau$, we can use equations (32)–(34) and equation (45) and prove that if $\hat{f}_{r,l,i}>\phi\tilde{C}_{r,l}\frac{1-\epsilon^{*}}{1+\epsilon^{*}}$, then, with probability at least $(1-\delta_{g})(1-e^{-d})$:

[TABLE]

Applying equation 45, we have that, if an item $i$ is selected to be reported in the set $H$, then, with probability at least $(1-e^{-d})$:

[TABLE]

Applying now equations 32-34, which are valid with probability at least $1-\delta_{g}$, we can state that, with probability at least $(1-\delta_{g})(1-e^{-d})$, i.e. with a probability of failure $\delta_{p}=\delta_{g}+e^{-d}(1-\delta_{g})$, if item $i$ is reported in the output set $H$, then:

[TABLE]

∎

Theorems 5 and 6 show that the set of candidate frequent items produced by P2PTFHH satisfies the requirements of the Approximate Time Faded Heavy Hitters problem (Problem 1). Thus, we can set the parameters of the algorithm, i.e. the dimensions of the sketches and/or the number of rounds to be executed, so that the absence of false negatives and the tolerance on false positives are guaranteed with certain probabilities of failure that in turn depend on the algorithm’s parameters and can be kept small.

Thus, the overall probability of failure of P2PTFHH is driven by $\delta_{p}$ and we can avoid dealing with the two distinct probabilities of failure, $\delta_{n}$ and $\delta_{p}$.

We know from Problem 1 that the desired value for the tolerance $\epsilon$ must be such that $\epsilon<\phi$; therefore, starting from the expression of $\epsilon$ from theorem 6, we have that:

[TABLE]

Under the condition above, we can see that $\delta_{n}<\delta_{p}$. In fact:

[TABLE]

We are now ready to state the following theorem, which follows straight from previous results and reasoning, and subsumes theorems 5 and 6.

Theorem 7**.**

Given an input stream $\mathcal{N}$, with total decayed count $C$, distributed among $p$ peers, each one making use of an augmented sketch data structure of size $d\times w$, and given a threshold parameter $0<\phi<1$, after $r$ rounds of P2PTFHH, on any peer the QUERY procedure returns a set $H$ of items and their corresponding time–faded frequencies so that, with probability greater than $1-\delta$:

$H$* includes all of the true time–faded heavy hitters ($f>\phi C$);* 2. 2.

$H$* does not include any item in $\mathcal{N}$ with frequency $f\leq(\phi-\epsilon)C$;*

*with a false positives tolerance $\epsilon=\frac{4\epsilon^{*}\phi}{(1+\epsilon^{*})^{2}}+\frac{e}{2w}\frac{1-\epsilon^{*}}{1+\epsilon^{*}}$,

and a probability of failure $\delta=\delta_{g}+e^{-d}(1-\delta_{g})$, $\delta_{g}$ being the probability of failure pertaining to the averaging gossip protocol.

5.6 Error bounds on the estimated frequency

The estimated frequencies provided by $\mathcal{D}_{r,l}$ refer to average frequencies. Thus, in order to obtain an estimate of the global frequency $f_{i}$ of an item $i$, we need to multiply $\hat{f}_{r,l,i}$ by $\tilde{p}_{r,l}$. From eqs. (32)–(36) and (45) we can compute the corresponding error bounds. The following theorem holds.

Theorem 8**.**

Given an input stream $\mathcal{N}$ with total decayed count $C$, distributed among $p$ peers, each one making use of an augmented sketch data structure of size $d\times w$, and given a threshold parameter $0<\phi<1$, after $r$ rounds of P2PTFHH, any peer can report a time–faded estimated frequency $f^{s}_{r,l,i}$ of an item $i\in[m]$ so that, with probability $1-\delta$:

[TABLE]

with $\delta=\delta_{g}+e^{-d}(1-\delta_{g})$, $\delta_{g}$ being the probability of failure pertaining to the gossip approximation.

Proof.

From equation (32), with probability greater than $1-\delta_{g}$:

[TABLE]

and from equation (33) and equation (34), with probability greater than $1-\delta_{g}$:

[TABLE]

Now, starting from eqs. (36), (45) and taking into account eqs. (52), (53), with probability greater than $(1-\delta_{g})(1-e^{-d})$, it follows that:

[TABLE]

Eventually, setting $f^{s}_{r,l,i}=\hat{f}_{r,l,i}\tilde{p}_{r,l}$, the relation (51) follows. ∎

5.7 Practical considerations

We conclude this Section discussing how to select proper values for the parameters $d$, $w$ and $R$, which represent respectively the number of rows and columns of the augmented sketch data structure and the minimum number of rounds required to solve the Approximate Time–Faded Heavy Hitters Problem with a given threshold $\phi$, tolerance $\epsilon$ and probability of failure $\delta$. Theorem 7 proves the correctness of the P2PTFHH algorithm, also providing the relation which ties the value of $\epsilon$ to the parameters of the algorithm.

Let us consider as fixed the values of $p^{*}$ and $\delta_{g}$, then, in order to guarantee that the desired false positives tolerance $\epsilon$ is not exceeded, the user can set the number of rounds $R$ and/or the number of sketch columns $w$. By fixing the value of $\epsilon$, the relation between $w$ and $R$ is given by equation (55).

[TABLE]

Among all of the possible values for $R$ and $w$, the user could follow a strategy oriented to maintain the number of rounds (hence the running time) as fewer as possible and to choose $w$ accordingly or vice-versa to maintain the number of counters (hence the space) as lower as possible and to choose $R$ accordingly. Let us now discuss both strategies.

With the first strategy, which can be called time-dominant, the user is interested on choosing $R$ and $w$ which guarantee a given $\epsilon$ such that $R$ is minimum. Equation (55) reveals that $R$ is a monotone decreasing function with $w$, hence the minimum value for $R$ is obtained when $w$ tends to infinity; moreover, since $w>0$ the minimum value for $R$ can be computed by imposing the following constraint:

[TABLE]

from which it follows that

[TABLE]

Since $R$ is an integer, the minimum value of $R$ is given by:

[TABLE]

Substituting the value of $R_{min}$ provided by equation (58) into equation (55) for $R$, it is possible to obtain the value for $w$.

With the second strategy, which can be called space-dominant, the user wants to keep the memory footprint as low as possible. Equation (55) reveals that $w$ is a monotone decreasing function with $R$, hence the minimum value for $w$ is obtained when $R$ tends to infinity. Evaluating equation (55) for $R\rightarrow\infty$ it holds that the minimum value for $w$ is given by:

[TABLE]

Considering that $w$ is an integer value,

[TABLE]

solving equation (55) by $\epsilon^{*}$ and using equation (60) it holds that:

[TABLE]

Since $\epsilon^{*}=p^{*}\sqrt{\frac{\gamma^{R}}{\delta_{g}}}$, it holds that:

[TABLE]

Since $R$ is an integer,

[TABLE]

Finally, we discuss how the parameters of the algorithm influence the probability of failure $\delta$. P2PTFHH is a probabilistic algorithm in which two sources of randomness occur: the first source of randomness is due to the way the information is spread over the peers at each round of the gossip protocol and is represented by the probability $\delta_{g}$; the second source of randomness is related to the mapping of an item to a cell of the augmented sketch data structure through the corresponding hash functions. The overall probability of failure of P2PTFHH in solving the Problem 1 with the desired guarantees can be controlled and made as small as desired, by acting on the parameter $d$, i.e. the number of rows of sketches, and the value of $\delta_{g}$, that in turn depends on the value of $R$.

Theorem 7 states that the Approximated Time–Faded Heavy Hitters Problem can be solved without false negative items and with a bounded number of false positive items with probability of failure less than $\delta=\delta_{g}+e^{-d}(1-\delta_{g})$. Thus, given a desired value of $\delta$, we have to choose $d$ such that:

[TABLE]

6 Experimental results

We implemented a simulator in C++ using the igraph library [16] in order to test the performance of our algorithm. The simulator has been compiled using the GNU C++ compiler g++ 4.8.5 on CentOS Linux 7. The tests have been performed on a machine equipped with two hexa-core Intel Xeon-E5 2620 CPUs at 2.0 GHz and 64 GB of main memory. The source code of the simulator is freely available for inspection and for reproducibility of results contacting the authors by email.

Objective of the experiments was to provide evidence of P2PTFHH’s behavior in various conditions, through the use of some useful and commonly referred metrics: Recall, Precision, and frequency estimation Error. Recall is defined as the fraction of frequent items retrieved by an algorithm over the total number of frequent items. Precision is the fraction of frequent items retrieved over the total number of items reported as frequent items candidates. The frequency estimation error that we measure and report is the Average Relative Error, ARE, committed in the frequency estimation of items outputted by the algorithm as frequent candidates. Relative Error is defined as usual as $\frac{\left|f^{S}-f\right|}{f}$, where $f^{S}$ is the frequency reported for an item and $f$ is its true frequency.

For every experiment, a global input stream of items has been generated following a Zipfian distribution (items are 32 bits unsigned integers) and each peer has been assigned a distinct part of that global stream, thus simulating the scenario in which each peer processes, independently of the other peers, its own local sub-stream. The peers collaboratively discover the frequent items of the union of their sub-streams. The experiments have been repeated 10 times setting each time a different seed for the pseudo-random number generator used for creating the input data. For each execution, we collected the peers’ statistics discussed above. Then, with reference to each peer, we determined the average value of those statistics over the ten executions. At last, we computed the mean and confidence interval of each statistics over all of the peers and plotted these values.

We fixed the size of the global stream at $100$ millions of items, and varied each one of the other parameters: the skew of the Zipfian distribution, $\rho$, the number of peers, $p$, the frequent items threshold, $\phi$, the width of the sketch used by each peer, $sw$, (while the depth is fixed to $4$) , and the fan-out, $fo$, setting non varying parameters to default values. Every experiment has been carried out by generating random P2P network topologies through both the Barabasi-Albert and the Erdos-Renyi random graphs models. Table 1 reports the sets of values (first row) and the default values (second row) used for each parameter. All of the plots representing the evolution of Recall and Precision are on linear-linear scale, whilst the plots that show the behaviour of the relative error are on linear-logarithmic scale.

Figures from 2 to 6 report the results of experiments carried out without taking into account peer’s churning. The Recall, Precision and the Average Relative Error measured when varying the skewness of the input disribution are respectively shown by Fig. 2a, Fig. 2b and Fig. 2c. Recall and Precision are always $100\%$, i.e. the algorithm is very robust with regard to skewness variations in the input. Moreover, we observe very low Average Relative Errors on frequency estimation, both in Barabasi-Albert graph (the worst) and Erdos-Renyi case (the better).

Figure 3 shows how P2PTFHH behaves with regard to variations of the threshold $\phi$. Also in this setting, Recall (Fig. 3a) is always $100\%$, Precision (Fig. 3b), on the other hand, presents a value slightly below $100\%$ only when $\phi=0.01$. Average Relative Errors (Fig. 3c) confirm what already told about the skewness plots.

Figure 4 plots the results of experiments where we varied the number of peers. Recall and Precision stay at the max value, whilst the Average Relative Error (Fig. 4c) slightly increases when the number of peers grows. Figure 4c also shows that the error decreases when the fan out increases from $1$ to $3$. This behavior is expected, with a greater fan out the number of messages exchanged in a round increases and information spreads quickly. If the number of rounds executed and the dimension of the sketches used by the peers is not adequate to the number of peers in the network, the metrics tend to get worse.

The plots related to the experiments in which we varied the width $sw$ of peers’ sketches (Figure 5) do not present a particular behavior in the interval of values tested, showing that in this case the sketch size used was always enough with regard to the number of rounds executed in order to guarantee a good accuracy.

A major sensitivity is exhibited by the algorithm when the number of rounds executed is varied, Figure 6. We note that the Precision grows and the Average Relative Error decreases as the number of rounds increases. This behavior is expected, given the theoretical analysis.

Overall the experiments show that our algorithm exhibits very good performance in terms of Recall, Precision, and Average Relative Error of the frequency estimation when the guidance of the theoretical analysis is taken into account in determining the size of sketches used by peers and the number of rounds to be executed. Furthermore, the algorithm proves to be very robust to variations in the skewness of the input dataset and the frequent items threshold.

Nonetheless, the results above refer only to the case of a static network. In the following, we show how the algorithm behaves in presence of churning.

6.1 Effect of the churn

The performance of our P2PTFHH algorithm has been tested also in more realistic contexts with the adoption of two churning models, the fail-stop model and the Yao model, proposed by Yao et al. [41].

In the fail-stop model, a peer could leave the network with a given failure probability and the failed peers can not join the network anymore.

In the Yao model, peers randomly join and leave the network. For each peer $i$, a random average lifetime duration $l_{i}$ is generated from a Shifted Pareto distribution with $\alpha=3$, $\beta=1$ and $\mu=1.01$. At the same way, a random average offline duration $d_{i}$ is generated from a Shifted Pareto distribution with same $\alpha$ and $\mu$ values and $\beta=2$. We recall here that if $X\sim\text{Pareto(II)}(\mu,\beta,\alpha)$, i.e., $X$ is a random variable with a Pareto Type II distribution (also named Shifted Pareto), than its cumulative distribution function is $F_{X}(x)=1-\left(1+\frac{x-\mu}{\beta}\right)^{-\alpha}$.

The values $l_{i}$ and $d_{i}$ are used to configure, for each peer $i$, two distributions $F_{i}$ and $G_{i}$. Distributions $G_{i}$ are Shifted Pareto distributions with $\beta=3$ and $\alpha=2d_{i}$. While, distributions $F_{i}$ can be both Pareto distributions with $\beta=2$, $\alpha=2l_{i}$, or exponential distributions with $\lambda=1/l_{i}$. Whenever the state of a peer changes, a duration value is drawn from one of the distributions, $F_{i}$ or $G_{i}$, based on the type of duration values (lifetime or offtime) which must be generated. We run our experiments with both the variants Pareto and Exponential lifetimes.

When using the fail-stop model, we run our algorithm with the default parameter values of Table 1 and varying the failure probability through the values: $0.0,0.01,0.05,0.1$. As shown in Figures 7a and 7b, the recall and precision metrics are not affected by the introduction of peer failures up to a failure probability equal to $0.1$. However, as expected, the average relative error on frequency estimations gets worse going from about $10^{-7}$ in case of no churn to about $10^{-1}$ when the failure probability is $0.1$, see Figure 7c.

When the Yao model of churning was adopted, we run our algorithm with the default values of Table 1 and the parameters of churning already discussed, varying the number of peers available to participate in the gossip and the number of rounds executed. Also in these cases, recall and precision are not affected by the introduction of churning. In fact, we obtained the same plots for recall and precision varying the number of peers and the number of rounds as those in Figures 4a, 4b and 6a and 6b, the reason why we do not report these plots again. On the other hand, the average relative error is affected by the churning, as expected: Figures 8a and 8b are related respectively to the ARE measured varying the number of peers and the number of rounds with Pareto distributions for lifetimes, meanwhile Figures 8c and 8d refer to the ARE measured when using Exponential distributions for lifetimes.

7 Conclusions

In this paper we presented the P2PTFHH (Peer–to–Peer Time–Faded Heavy Hitters) distributed algorithm. Building on the distributed averaging protocol, we designed a gossip–based version of our sequential FDCMSS (Forward Decay Count-Min Space-Saving) algorithm for distributed mining of time–faded heavy hitters. To the best of our knowledge, this is the first distributed protocol designed specifically for the problem of mining on unstructured P2P networks time–faded heavy hitters, a data mining task with wide applicability, for instance in the context of sensor data mining, for business decision support, analysis of web query logs, network measurement, monitoring and traffic analysis. In these applications, the use of the time–fading model discounts the effect of old data, by putting heavier weights on recent batches of streaming data than older batches. We formally proved the algorithm’s correctness and error bounds on frequency estimation. The extensive experimental results also proved that P2PTFHH is extremely accurate and fast, allowing near real time processing of large datasets.