Relaxing Concurrent Data-structure Semantics for Increasing Performance: A Multi-structure 2D Design Framework

TL;DR

This paper introduces a novel two-dimensional framework for designing lock-free, concurrent data structures that relax semantics to enhance scalability and performance, demonstrating significant improvements over existing methods.

Contribution

It presents a flexible, monotonic relaxation framework using multiple data structure instances and operation grouping to improve concurrent data structure performance.

Findings

Framework achieves higher scalability and throughput.

Monotonically increasing performance with relaxation levels.

Outperforms existing concurrent data structures in experiments.

Abstract

There has been a significant amount of work in the literature proposing semantic relaxation of concurrent data structures for improving scalability and performance. By relaxing the semantics of a data structure, a bigger design space, that allows weaker synchronization and more useful parallelism, is unveiled. Investigating new data structure designs, capable of trading semantics for achieving better performance in a monotonic way, is a major challenge in the area. We algorithmically address this challenge in this paper.To address this challenge, we present an efficient lock-free, concurrent data structure design framework for out-of-order semantic relaxation. Our framework introduces a new two dimensional algorithmic design, that uses multiple instances of an implementation of the given data structure. The first dimension of our design is the number of data structure instances onto which operations are spread to, in order to achieve increased parallelism through disjoint memory accesses. The second dimension is the number of consecutive operations of a single thread that can stay at the same data structure instance in order to benefit from data locality. Our design can flexibly explore this two-dimensional space to achieve the property of monotonically increasing throughput performance via relaxing concurrent data structure semantics within a tight deterministic relaxation bound, as we prove in the paper. We show how our framework can instantiate lock-free out-of-order queues, stacks, counters and dequeues. The experimental evaluation shows that our two-dimensional data structures: i) significantly outperform the respected previous proposed ones with respect to scalability and throughput performance and ii) monotonically increase throughput as relaxation increases.

Figures

Tables

Table 1. Table 1 . 2 ​ D 2 𝐷 2D derived data structure algorithms

Algorithm	WinCoupled	WinDecoupled
2D-Stack	2Dc-Stack	2Dd-Stack
2D-Queue	—	2Dd-Queue
2D-Counter	2Dc-Counter	2Dd-Counter
2D-Deque	—	2Dd-Deque

Table 2. Table 2. Execution parameter configuration for the different algorithms

Algorithm	0pt
Random (Stack,Counter, Queue & Deque)	$3P$
Random-C2 (Stack,Counter & Queue)	$3P$
S-robin	$\left(k/\left(2P-1\right)\right)+1$
C-robin	$\left(k/\left(2P-1\right)\right)+1$
Q-robin	$\left(k/\left(P-1\right)\right)+1$
Deque-robin	$\left(k/\left(2P-1\right)\right)+1$
k-Stack	$k+1$
Q-segment	$k+1$
lru	$k+1$
2Dd-Stack	$\left(k/\left(30pt\right)\right)+1$
2Dc-Stack	$\left(k/\left(2shift+0pt\right)\right)+1$
2Dd-Queue	$\left(k/0pt\right)+1$
2Dd-Deque	$\left(k/30pt\right)+1$
2Dc-Counter	$\left(k/\left(shift+0pt\right)\right)+1$
2Dd-Counter	$\left(k/20pt\right)+1$

Equations

$$\{\begin{array}{ll}\mathbb{P}\left({\mathcal{S}}_i\to {\mathcal{S}}_{i+1}\right)=p,& \text{if }0<i<\text{K}\\ \mathbb{P}\left({\mathcal{S}}_i\to {\mathcal{S}}_{i-1}\right)=1-p,& \text{if }0<i<\text{K}\\ \mathbb{P}\left({\mathcal{S}}_i\to {\mathcal{S}}_{\text{K}-(shift\times 0pt+1)}\right)=p,& \text{if }i=\text{K}\\ \mathbb{P}\left({\mathcal{S}}_i\to {\mathcal{S}}_{(shift\times 0pt-1)}\right)=1-p,& \text{if }i=0\\ \mathbb{P}\left({\mathcal{S}}_i\to {\mathcal{S}}_j\right)=0,& \text{otherwise}\end{array}$$

$$\mathbb{E}\left(hop|\text{Put}\right)=\sum _{i=0}^{\text{K}}{\pi }_i\mathbb{E}\left(hop|{\mathcal{S}}_i,\text{Put}\right)<(\sum _{f=0}^{0pt-1}\frac{0pt}{0pt-f})\frac{2p}{\text{K}+1}<(\ln (0pt-1)+\gamma )\frac{0pt}{\text{K}+1}<(\ln 0pt+\gamma )\frac{1}{0pt}$$

Full text

To address this challenge, we present an efficient lock-free, concurrent data structure design framework for out-of-order semantic relaxation. Our framework introduces a new two dimensional algorithmic design, that uses multiple instances of an implementation of the given data structure. The first dimension of our design is the number of data structure instances onto which operations are spread to, in order to achieve increased parallelism through disjoint memory accesses. The second dimension is the number of consecutive operations of a single thread that can stay at the same data structure instance in order to benefit from data locality. Our design can flexibly explore this two-dimensional space to achieve the property of monotonically increasing throughput performance via relaxing concurrent data structure semantics within a tight deterministic relaxation bound, as we prove in the paper.

We show how our framework can instantiate lock-free out-of-order queues, stacks, counters and dequeues. We provide implementations of these relaxed data structures and evaluate their performance and behaviour on two parallel architectures. The experimental evaluation shows that our two-dimensional data structures: i) significantly outperform the respected previous proposed ones with respect to scalability and throughput performance and ii) monotonically increase throughput as relaxation increases.

semantics relaxation, data structures, concurrency, lock-freedom, performance, scalability, stack, deque, queue, counter

††copyright: none

1. Introduction

Concurrent data structures allow operations by multiple threads to concurrently access the data structures, and this requires synchronisation to guarantee consistency with respect to their sequential semantics (dijkstra1965solution, ; Dijkstra:1968:SLS:363095.363143, ). The synchronisation of concurrent accesses aims at enforcing some notion of atomicity, where, an operation appears to occur at a single instant between its invocation and its response. A concurrent data structure is typically designed around one or more synchronisation access points, from where threads compute, consistently, the current state of the data structure. Synchronisation is vital in achieving consistency and cannot be eliminated (Attiya:2011:LOE:1925844.1926442, ). Whereas this is true, synchronisation might generate contention in memory resources hurting scalability and performance.

The necessity of reducing contention at the synchronisation access points, and consequently improving scalability, is and has been a major focus for concurrent data structure researchers. Aiming to address this challenge, techniques like: elimination (afek2010scalable, ; hendler2010scalable, ; shavit1995elimination, ), combining (shavit2000combining, ), dynamic elimination-combining (DBLP:journals/corr/abs-1106-6304, ), back-off, operation buffers and batching have been proposed. To address, in a more significant way, the challenge of scalability bottlenecks of concurrent data structures, it has been proposed that the semantic legal behaviour of data structures should be extended (Shavit:2011:DSM:1897852.1897873, ). This line of research has led to the introduction of an extended set of weak semantics including weak internal ordering, weakening consistency and semantic relaxation.

One of the main definitions of data structure semantic relaxation, proposed and used in the literature, is k-out-of-order (afek2010quasi, ; henzinger2013quantitative, ; Talmage2017, ; Haas:2013:DQS:2482767.2482789, ; rihani2015brief, ; wimmer2015lock, ). k-out-of-order semantics allow operations to occur out of order within a given $k$ bound, e.g. a pop operation of a k-out-of-order stack can remove any item among the $k$ topmost stack items. By allowing a Pop operation to remove any item among the $k$ topmost stack items, the extended stack semantics do not anymore impose by definition a single access point for all its operations. Thus, allowing for potentially more efficient stack designs with lower synchronisation overhead.

Semantics relaxation can be exploited to better use disjoint access points to achieve improved parallelism and thread local data processing (DBLP:conf/wdag/RukundoAT19, ). Disjoint access is popularly achieved by distributing data structure operations over multiple disjoint instances of a given data structure (Haas:2013:DQS:2482767.2482789, ; rihani2015brief, ; DBLP:conf/concur/HaasHHKLPSSV16, ; DBLP:conf/esa/Williams0D21, ). On the other hand, the locality is generally achieved through letting a thread stick to the same memory location for a given number of consecutive operations (wimmer2015lock, ; gidron2012salsa, ; DBLP:conf/concur/HaasHHKLPSSV16, ).

In this paper, we introduce an efficient two-dimensional algorithmic design framework, that uses disjoint multiple instances (sub-structures) of a given data structure as shown in Figure 1. The first dimension of the framework is the number of sub-structures operations are spread to, in order to benefit from parallelism through disjoint access points. The second dimension is the number of consecutive operations that can occur on the same sub-structure in order to benefit from data locality by a thread sticking to the same memory location for a given number of operations. We use two parameters to control the dimensions; 0ptfor the first dimension (horizontal) and 0ptfor the second dimension (vertical) as shown in Figure 1.

A thread can operate on a given sub-structure for as long as a set of conditions hold for this sub-structure with respect to the operation on hand, in this case we call the sub-structure valid. As an example, a valid sub-structure might be one that does not exceed max or go below min, as depicted by the dashed green rectangle in Figure 1. Validity conditions make sub-structures valid or invalid for a given operation. This implies that threads have to search for a valid sub-structure for the respective operation, increasing operation cost (latency). Our framework efficiently controls this search overhead by limiting the number of sub-structures utilised, and by allowing a thread to operate on the same sub-structure consecutively for as long as the validity conditions hold. Using these validity conditions, our framework generates tenable and tunable relaxation behaviour, tightly bound by a deterministic relaxation bound.

Our algorithmic design framework can be used to extend existing lock-free data structure algorithms to achieve k-out-of-order semantics. This can be achieved with minimal modifications to the original data structure algorithm as we later show in Section 6. Using our framework, we derive semantically relaxed lock-free k-out-of-order stacks, queues, dequeue and counters together with proof of correctness. Experimental evaluation shows that the derived data structures significantly outperform all respective previous known data structure implementations and monotonically increase throughput as relaxation increases.

The rest of the paper is structured as follows. In Section 2 we discuss the literature related to this work. We present the $2D$ framework in Section 3 followed by the optimisation discussion in Section 4 and complexity analysis in Section 5. The derived data structure algorithms are presented in Section 6. With the aim to enrich the evaluation of our algorithmic design, a pallet of additional multi structure algorithms are presented in Section 8. An experimental evaluation is presented and discussed in Section 9. The paper concludes in Section 10.

2. Related Work

Recently, data structure semantic relaxation has attracted the attention of researchers, as a promising direction towards improving concurrent data structures’ scalability (henzinger2013quantitative, ; Talmage2017, ; Shavit:2011:DSM:1897852.1897873, ; DBLP:conf/esa/Williams0D21, ). It has also been shown that small changes on the semantics of a data structure can have a significant effect on the computational power of the data structure (Shavit:2016:CRD:3011492.3011515, ). The interest in semantic relaxation is further supported by the need to optimise concurrent data structures for the ever growing parallel/multi-core computing hardware. As discussed earlier in Section 1, one of the main definitions of semantic relaxation proposed and used in the literature is k-out-of-order.

Using the k-out-of-order definition, a segmentation technique has been proposed in (afek2010quasi, ), later revisited in (henzinger2013quantitative, ) realising a relaxed Stack (k-Stack) and FIFO Queue (Q-segment) both with k-out-of-order semantics. The technique involves a linked-list of memory segments with $k$ number of indexes on which an item can be added or removed per index. The stack items are accessed through the topmost segment, whereas the queue has a tail and head segment from which Enqueue and Dequeue can occur respectively. Segments can be added and removed. Relaxation is only controlled through varying the number of indexes per segment. As discussed in Section 1, increasing the number of indexes increases operation latency and later becomes a performance bottleneck. This limits the performance benefits of the technique to a small range of relaxation values.

Also, load balancing together with multiple queue instances (sub-queues) has been used to design a relaxed FIFO queue (lru) with k-out-of-order semantics (Haas:2013:DQS:2482767.2482789, ). Each sub-queue maintains two counters, one for Enqueue another for Dequeue, while two global counters, one for Enqueue and one for Dequeue maintain the total number of operations for all sub-queues. The global counters are used to calculate the expected number of operations on the least-recently-used sub-queue. Threads can only operate on the least-recently-used sub-queue. This implies that for every operation, threads must synchronise on the global counter, making it a sequential bottleneck. Moreover, threads have to search for the least-recently-used sub-queue leading to high operation and memory latency due to lack of thread locality.

Random distribution of data structure operations over multiple instances (sub-structures) of a given data structure has been used to design relaxed priority queues (rihani2015brief, ), and has been tested on other data structures including stacks, counters and FIFO queues (DBLP:conf/wdag/RukundoAT19, ). For each operation, a thread randomly selects a sub-structure onto which it goes ahead to perform that given operation. To improve on the quality (order guarantees), a given thread can select a set of sub-structure uniformly at random, and then selects an appropriate sub-structure (according to the data structure semantics) from the given set of sub-structures onto which the thread proceeds to operate. The technique significantly reduces contention where the number of sub-structures is higher than the number of threads and operations are distributed uniformly at random. However, as discussed in (DBLP:conf/spaa/RukundoAT22, ), as the number of sub-structures increase, memory latency increases due to lack of thread locality and limits the performance benefits gained from reduced contention. In terms of k-out-of-order, the technique does not provide for a deterministic relaxation bound ($k$) but rather a probabilistic one under specific assumptions (Alistarh:2017:PCP:3087801.3087810, ).

The proposed relaxation techniques, mentioned above, apply relaxation in one dimension, i.e, increase the number of disjoint access points to improve parallelism and reduce contention. However, this also increases operation latency due to the increased number of access points to select from, and memory latency due to lack of locality. Without a remedy to this downside, the proposed techniques cannot provide monotonic relaxation for better performance. Towards addressing the challenge of locality, some designs such as that of a priority (rihani2015brief, ) have been revisited (DBLP:conf/esa/Williams0D21, ) to introduce thread locality. The authors noted that a thread accesses several cache lines from randomly chosen sub-structures which are rarely reused but usually cause cache invalidation costs later, a downside also studied and discussed in (DBLP:conf/spaa/RukundoAT22, ). To improve locality (DBLP:conf/esa/Williams0D21, ), three measures are applied; insertion and deletion buffers, operation batching and a thread sticking on the same set of sub-structures for several consecutive operations. Operation buffers and batching have been frequently used in the literature to reduce contention at a specific memory location (10.1145/3210377.3210388, ; alistarh2015spraylist, ; 10.1007/978-3-030-85665-6_24, ; wimmer2015lock, ). Buffering and batching can also help to improve locality where threads can have exclusive buffers (similar to sticking to a given memory location) (wimmer2015lock, ), otherwise, the technique would also suffer from memory latency due to cache invalidation costs. One of the most efficient techniques for improving thread locality is having a thread operate on the same memory location (sticking) for several consecutive operations to avoid cache line sharing and invalidation (DBLP:conf/spaa/RukundoAT22, ).

Although thread memory sticking can improve locality, it can significantly reduce the quality (order guarantees) of a relaxed data structure, outweighing the performance gained from locality as observed in (DBLP:conf/esa/Williams0D21, ). As the thread stickiness increases, the expected quality drops at a higher rate than the measured increase in throughput performance. It is also noted that thread stickiness under random distribution complicates the provision of order guarantees. In other works, the thread buffer technique has been used to realise a relaxed priority queue (wimmer2015lock, ). Here each thread is assigned a local buffer (a sticky sub-structure) onto which it performs its priority queue operations, later when the buffer is full, the given thread flushes its buffer to a shared priority queue. On the other hand when the thread buffer is empty, the given thread can get (work stealing) items from other available thread buffers or get items from the shared priority queue. In this case, thread buffers provides a deterministic relaxation bound, however, the bound is a product of the degree of stickiness (number of operations per buffer) and number of threads. This implies that an increase in number of threads or buffer size leads to a drop in data quality, facing the same challenge as discussed above. Buffer techniques can face a challenge of merging data especially for priority queues where keys might have to be inserted independently at different points within the data structure as observed in (wimmer2015lock, ). As noted in (wimmer2015lock, ) work stealing can increase the complexity of the data structure design and also reduce cache efficiency since it involves spying and modifying other thread buffers.

Without using sub-structures, a relaxed skip list based priority queue is proposed, where the delete operations are randomly distributed (sprayed) to different points within a skip list (alistarh2015spraylist, ). The delete operations avoid the sequential bottleneck at the head by randomly walking the list and returning any element among the first $O(plog^{3}p)$, where $p$ is the number of threads. The technique reduces contention that can arise from concurrent delete operations trying to access and return the same element at the head. The technique provides a probabilistic relaxation bound that is proportional to the number of threads. This implies, that trying to increase throughput through increasing number of threads reduces the quality of the data.

Apart from semantic relaxation, other design strategies such as internal weak ordering (Dodds:2015:SCT:2775051.2676963, ), and local linearizability (DBLP:conf/concur/HaasHHKLPSSV16, ) have been proposed to extend semantics for improving scalability. Weak internal ordering has been proposed and used to implement a timestamped stack (TS-Stack) (Dodds:2015:SCT:2775051.2676963, ), where Push timestamps each pushed item. Each thread has its local buffer onto which it performs Push operations. However, Pop operations pay the cost of searching for the latest item. In the worst case, Pop operations might contend on the same latest item if there are no concurrent Push operations. This leads to search retries, especially for workloads with higher Pop rates (bursts) than Push ones.

Local linearizability has also been proposed for designing efficient concurrent data structures such as; FIFO queues and Stacks (DBLP:conf/concur/HaasHHKLPSSV16, ). Locally linearizable FIFO queues and Stacks proposed in (DBLP:conf/concur/HaasHHKLPSSV16, ) relies on multiple instances of a given data structure. Each thread is assigned an instance on which it locally linearizes all its operations. Operations: Enqueue (FIFO queue) or Push (Stack) occur on the assigned instance for a given thread, whereas, Dequeue or Pop can occur on any of the available instances. With Dequeue or Pop occurring more frequently, contention quickly builds as threads try to access remote buffers. The threads also lose the locality advantage while accessing remote buffers, cancelling out the caching advantage especially for single access data structures such as the Stack (Hackenberg:2009:CCA:1669112.1669165, ; David:2013:EYA:2517349.2522714, ; Schweizer:2015:ECA:2923305.2923811, ).

Without relaxing or weakening consistence of the data structure sequential semantics, elimination (afek2010scalable, ; shavit1995elimination, ; DBLP:journals/corr/abs-1106-6304, ) and combining (shavit2000combining, ) have also been proposed as way to improve scalability by reducing contention at specific points within a given data structure. Elimination implements a collision path on which different concurrent operations (such as Push and Pop for the case of a stack) try to collide and cancel out, without accessing a joint access point (such as a stack’s top), otherwise, they proceed to access the central structure (hendler2010scalable, ; Moir:2005:UEI:1073970.1074013, ). Such operation pairs create disjoint collisions that can be executed in parallel. Combining, on the other hand, allows operations from multiple threads to be combined and executed by a single thread without the other threads contending on the central structure (Fatourou:2012:RCS:2145816.2145849, ; Hendler:2010:FCS:1810479.1810540, ). However, their performance depends on the specific workload characteristics. Elimination mostly benefits symmetric workloads, whereas combining mostly benefits asymmetric workloads. Furthermore, the central structure sequential bottleneck problem still persists.

3. The 2D Framework

In this section, we describe our disjoint multi-structure $2D$ design framework. The 2D framework uses disjoint multiple instances (sub-structures) of the given data structure as depicted in Figure 1. Threads can select and operate on any of the sub-structures following a maximum and minimum operation count threshold. Herein, operation refers to the process that updates the data structure state by adding (Put) or removing (Get) an item (Push and Pop respectively for the stack example). Each sub-structure holds a counter (sub-count) that counts the number of local successful operations.

A combination of operation count threshold and number of sub-structures, form a logical count period, we refer to it as Window, depicted by the dashed green rectangle in Figure 1. The Window limits the number of operations that can occur on each sub-structure ([\textit{Win_{max}},\textit{Win_{min}}]): maximum ($Win_{max}$) and minimum ($Win_{min}$) operation count threshold for all sub-structures, for a given period. This implies, that for a given period, a sub-structure can be operated on (valid) or not (invalid) as exemplified in Figure 1, and a Window can be full or empty. The Window is full if all sub-structures have the maximum number of operations (\textit{sub-count}=\textit{Win_{max}}), empty, if all sub-structures have the minimum number of operations (\textit{sub-count}=\textit{Win_{min}}). The Window is defined by two parameters; 0ptand 0ptas exemplified in Figure 1. $0pt=\#\textit{sub-structures}$, and 0pt=\textit{Win_{max}}-\textit{Win_{min}}.

To validate a sub-structure, its sub-count is compared with either $Win_{max}$ or $Win_{min}$ depending on the operation; \textit{sub-count}<\textit{Win_{max}} or \textit{sub-count}>\textit{Win_{min}}. If the given sub-structure is invalid, the thread has to hop to another sub-structure until a valid sub-structure is found (validity is operation specific as we discuss later). If a thread cannot find a valid sub-structure, then, the Window is either full or empty. The thread will then, either increment or decrement both $Win_{max}$ and $Win_{min}$, the process we refer to as, Window shifting. A Window can shift up or down, and is controlled by $shift_{up}$ or $shift_{down}$ values respectively, where, 0<\textit{shift_{up}},\textit{shift_{down}}\leq 0pt. The thread can only shift the Window by a given shift value. $shift_{up}$ and $shift_{down}$ values can be configured differently to optimise for different workloads and execution environment.

Relaxation is controlled by the two parameters that define a Window: 0ptand 0pt. 0ptand 0ptprovide a deterministic relaxation bound for the derived k-out-of-order relaxed data structure as proved in Section 7. We define and present the implementation of two types of Windows: WinCoupled (2Dc) and WinDecoupled (2Dd). WinCoupled implements a single access Window for all data structure operations, whereas WinDecoupled implements an independent Window for each data structure operation.

3.1. WinCoupled

WinCoupled couples both Put and Get to share the same Window and sub-count for each sub-structure. A successful Put increments whereas, a successful Get decrements the given sub-count. On a full Window, Put increments $Win_{max}$ shifting the Window up by a given value ($shift_{up}$), whereas, on an empty Window, Get decrements $Win_{max}$, shifting the Window down by a given value ($shift_{down}$). WinCoupled resembles elimination (shavit1995elimination, ), only that here, we cancel out operation counts for matching Put and Get on the same sub-structure within the same Window. Just like elimination reduces joint access updates, WinCoupled reduces Window shift updates.

In Algorithm 1, we present the algorithmic steps for WinCoupled. Recall, $0pt=\#\textit{sub-structures}$ and 0pt=\textit{Win_{max}}-\textit{Win_{min}}. Each sub-structure is uniquely identified by a descriptor that holds information including a pointer to the sub-structure, sub-count counter, and a version number (Line 1 to 1). The version number is to avoid ABA111The ABA problem occurs when multiple processors access a shared location without noticing each others changes. related issues, where different sub-structure states might hold same descriptor update values. Using a wide CAS, we update the descriptor information in a single atomic step (Line 1). This helps maintain the integrity of the underlying sub-structures with minimal structural modifications. We use an array of descriptors through which threads can access any of the available sub-structures.

To perform an operation, the thread has to search and select a valid sub-structure within a Window period. At the beginning of every search, the thread stores a copy of the current Window locally (Line 1) which is used to detect Window shifts while searching (Line 1, 1). During the search, the thread validates each sub-structure count against $Win_{max}$ (Line 1, 1). If no valid sub-structure is found, $Win_{max}$ is updated atomically, shifting the Window up or down (Line 1). Put increments $Win_{max}$ to shift the Window up (Line 1), whereas, Get decrements $Win_{max}$ to shift the Window down (Line 1). Before index hopping or Window shifting, the thread must verify that the Window has not yet shifted (Line 1 or 1 respectively). For every Window shift during the search, the thread restarts the search with the new Window (Line 1, 1).

If a valid sub-structure is selected, the respective descriptor state (Line 1) and index are returned (Line 1, 1). The thread can then proceed to try and operate on the given sub-structure using the respective descriptor information. As a way to check if the data structure is empty, the Window search can only return an empty sub-structure (Line 1), if during the search, all sub-structures where empty ($NULL$ pointer) (Line 1). Using the Window parameters, 0pt, and 0pt, we can tightly bound the relaxation behaviour of the derived 2Dc data-structure as discussed later in Section 7.

However, we should note that, Window and sub-structure updates occur independent of each other. For WinCoupled, this can lead to a sub-structure being updated although the Window it was selected from has since shifted. Take as an example, a Get selecting a sub-structure from a full Window ($w_{1}$) at time ($t_{1}$), followed by a Put that reads the full Window and shifts it up to $w_{2}$ at $t_{2}$. It is possible for the Get to update the selected sub-structure at $t_{3}$ based on $w_{1}$ that has since shifted to $w_{2}$. This difference is however bounded as proved in Section 7.

3.2. WinDecoupled

WinDecoupled decouples the data structure operations (Put and Get) and assigns them independent Windows. Also, an independent sub-count is maintained for each data structure operation, on each sub-structure. Unlike WinCoupled, each successful operation increments its respective sub-count and $Win_{max}$ on a full Window. This implies that both sub-count and Window counters increase monotonically as shown in Algorithm 2.

Recall that operations are decoupled to operate within independent Windows (Line 2, 2). Therefore, the sub-structure descriptor differs from that used for WinCoupled; it includes a pointer to the sub-structure, Get-sub-count counter for Get operations (Line 2) and Put-sub-count counter for Put operations (Line 2). Note that we do not need a version number since the counter updates monotonically increment it, which avoids the ABA problem. The descriptor can be updated in one atomic step using a wide CAS instruction (Line 2, 2), just like in WinCoupled. Unlike WinCoupled, WinDecoupled can use one or more arrays of descriptors where necessary. As an example, we describe the WinDecoupled using two arrays of descriptors through which threads can access any of the available sub-structures; Put descriptor’s for threads performing Put operations, and Get descriptor’s for threads performing Get operations. Apart from accessing a different array for each operation, the Window search steps are similar to those of WinCoupled.

To perform an operation, the thread has to search and select a valid sub-structure within a Window period for the given operation. The type of operation determines which descriptor array to access (Line 2, 2) and Window to operate with (Line 2 or 2). At the beginning of every search, the thread stores a copy of the Window locally (Line 2, 2) which is used to detect Window shifts while searching (Line 2, 2, 2, 2). During the search, the thread validates each sub-structure against the given $Win_{max}$ specific to the operation (Line 2, 2).

If no valid sub-structure is found, the $Win_{max}$ is incremented, shifting the Window up. Both Put and Get shift the Window up (Line 2, 2) by incrementing the $Win_{max}$ count by 0pt(Line 2, 2). Before Window shifting or hopping to another index, the thread has to confirm that the Window has not shifted from the locally known state (Line 2, 2). This guarantees that threads always start their search within the most current Window for the respective operation. For every Window shift during the search, the thread restarts the search with the new Window (Line 2, 2).

If a valid sub-structure is selected, the respective descriptor state (Line 2, 2) and index are returned (Line 2, 2). The thread can then proceed to try and operate on the given sub-structure using the descriptor information. As a way to check if the data structure is empty, the Window search can only return an empty sub-structure (Line 2), if during the search, all sub-structures where empty ($NULL$ pointer) (Line 2). Just like WinCoupled, we use the Window parameters, 0pt, and 0pt, to tightly bound the relaxation behaviour of the derived 2Dd data-structures as we will discuss in Section 7.

4. Optimisations

Our multi-structure 2D design framework can be tuned to optimise for; locality, contention and sub-structure search overhead (hops), using the 0ptand 0ptparameters.

Locality occurs when a thread accesses and operates on the same valid sub-structure consecutively before another thread operates on the given sub-structure. This implies that, the data corresponding to the access-point of the given sub-structure will mostly be maintained in the local memory of the thread operating on the sub-structure for the given number of consecutive operations.

Contention occurs when more than one thread try to access and operate on the same valid sub-structure at the same time, in other words, threads will contend for the given valid sub-structure. In our case where synchronisation is implemented using CAS, only one of the threads will succeed to operate on the sub-structure, while the other contending threads will have to retry. Each thread maintains a contention variable (contention) that is set to true when a thread fails to operate on a valid sub-structure due to a failed CAS, signalling the presence of contention.

A hop occurs when a thread tries to access an invalid sub-structure. In this case, the thread has to hop from the invalid sub-structure to another sub-structure while searching for a valid sub-structure. Hops increase as the number of invalid sub-structures increases. Note that this is different from when a thread fails on a given sub-structure due to contention and retries by searching for another valid sub-structure.

4.1. Locality

To exploit locality, the thread starts its sub-structure search from the previously known sub-structure index on which it succeeded. This allows the thread a chance to operate on the same sub-structure multiple times locally, given that the sub-structure is still valid. To further improve on locality, a thread that fails on a selected sub-structure due to contention (failed CAS), randomly searches for another sub-structure (Line 1, 2) leaving the successful thread to take over the selected sub-structure locally. The thread will operate on the same sub-structure locally, for as long as it does not fail due to contention and the sub-structure is valid. Using the 0ptparameter, we can control the number of consecutive data structure operations that can be performed on a sub-structure for a given Window period, consequently allowing us to tune locality.

Working locally improves the caching behaviour of the system by reducing remote memory access, consequently improving throughput performance (Hackenberg:2009:CCA:1669112.1669165, ; David:2013:EYA:2517349.2522714, ; Schweizer:2015:ECA:2923305.2923811, ). Locality also reduces the number of hops as discussed in Section 5 and supported by Theorem 2 and 3. The cost of hopping includes reading new memory locations, which introduces higher memory latency and cache coherence costs. Reducing the number of hops improves throughput performance especially under a NUMA execution environment with high communication cost between NUMA nodes (Hackenberg:2009:CCA:1669112.1669165, ; David:2013:EYA:2517349.2522714, ; Schweizer:2015:ECA:2923305.2923811, ).

4.2. Contention

A failed operation on a valid sub-structure signals the possibility of contention. The thread that fails on a valid sub-structure, starts the sub-structure search on a randomly selected index (Line 1, 2). This reduces possible contention that might arise if the failed threads were to retry on the same sub-structure. Furthermore, random selection avoids contention on individual sub-structures by uniformly redistributing the failed threads to all available sub-structures.

At the beginning of the sub-structure search (Line 1, 2, 2), if the search start index has an invalid sub-structure, the thread tries a given number of uniformly distributed random hops (Line 1, 2) before switching to round robin hops (Line 1, 2) until a valid sub-structure is found. The random hops evenly distributes the threads to avoid contention and when a valid sub-structure is not found in the initial random hops, threads end up with randomly distributed starting points for the round robin search, further avoiding contention that can arise from threads contending on the same sub-structures during the search. In our implementation we use two random hops as the optimal number for a random search, based on the power of random two choices result (mitzenmacher2001power, ). However, this is a configurable parameter that can take any value.

We further note that contention is inversely proportional to the 0pt. As a simple model, we split the latency of an operation into a contention ($op_{cont}$) and a contention-free ($op_{free}$) operation cost, estimated by the following formula: $op=op_{cont}/0pt+op_{free}$. This means that we can increase the 0ptto further reduce contention when necessary.

4.3. Hops

The number of hops increases with an increase in 0ptdue to the increase in possible numbers of invalid sub-structures. This counteracts the performance benefits from contention reduction through increasing 0pt, necessitating a balance between contention and hops reduction. Based on our simple contention model above ($op=op_{cont}/0pt+op_{free}$), the performance would increase as the contention factor vanishes with the increase of $0pt$, but with an asymptote at $1/op_{free}$. This implies that beyond some point, one cannot really gain throughput by increasing the 0pt, however, throughput would get hurt due to the increased number of hops. At some point as 0ptincreases, gains from the contention factor ($\lim_{0pt\to\infty}op_{cont}\to 0$) are surpassed by the increasing cost of hops. This is something that we want to avoid in our effort of relaxing semantics for gaining throughput. To avoid this, we switch to increasing 0ptinstead of 0pt, at the point of 0ptsaturation. Increasing 0ptreduces the number of hops. This is supported by our step complexity analysis presented in Section 5 Theorem 2 and 3.

WinCoupled can further be optimised to minimise the number of search hops that arise from shifting the Window. A thread performing a Put operation will shift the Window up if all available sub-structures have a maximum number of operation count. On the other hand, a thread performing a Get operation will shift the Window down if all available sub-structures have a minimum number of operation count given that the data structure is not empty. Shifting the Window in intervals of 0pt(\textit{shift^{up}}=0pt), shifts the Window up to an empty state where all sub-structures will have the minimum operation count for the given Window, or shifts the Window down to a full state where all sub-structures will have the maximum operation count for the given Window. Shifting the Window up invalidates all sub-structures for Get operations at that time (Line 1), while shifting the Window down invalidates all sub-structures for Put operations at that time (Line 1). To reduce sub-structure invalidation on Window shifts and subsequently minimise the number of hops on both full and empty Window states, we configure the Window to shift with respect to the operation rate. This implies that, if $\textit{Push}rate>\textit{Pop}rate$ then \textit{shift^{up}}>\textit{shift_{down}} where \textit{shift^{up}}+\textit{shift_{down}}=0pt. With this configuration, the Window will shift with a bigger interval for the operation with a higher rate, giving threads performing that operation a higher chance to find a valid sub-structure without invalidating sub-structures for the other operation. This reduces hops and also enhances locality.

5. Complexity Analysis

In this section, we analyse the correlation between hops and the Window parameters 0ptand 0pt. We provide the expected step complexity of our framework by considering the sequential executions of our algorithm; where a single thread executes a sequence of operations. The type of the operations in the sequence is determined independently with a fixed probability, where $p$ denotes the probability of a Put operation.

5.1. WinCoupled

Recall $Win_{max}$ regulates the maximum number of operations per sub-structure within a given Window. Also recall that 0pt= #sub-structures. Let the number of operations of a sub-structure $i$ be given by $N_{i}$. Put operations are allowed to occur at sub-structure $i$, if \textit{N_{i}}\in[\textit{Win_{max}}-0pt,\textit{Win_{max}}-1], whereas Get operations are allowed to occur at sub-structure $i$, if \textit{N_{i}}\in[(\textit{Win_{max}}-0pt)+1,\textit{Win_{max}}]. This basically means that, at any time, the number of operations of a sub-structure can only variate in the vicinity of $Win_{max}$, more precisely: \forall i,(\textit{Win_{max}}-0pt)\leq\textit{N_{i}}\leq\textit{Win_{max}}. We refer to this interval as the active region of the sub-structure.

We introduce the random variable \textit{N^{active}{i}}=\textit{N{i}}-(\textit{Win_{max}}-0pt) where \textit{N^{active}{i}}\in[0,0pt] that provides the number of items in the active region of the sub-structure $i$ and the random variable \textit{N^{active}}=\sum_{i=1}^{0pt}\textit{N^{active}{i}} that provides the total number of items in the Window.

As mentioned before the 0ptdimension tries to exploit locality, thus, a thread starts an operation with a query on the sub-structure where the given thread’s last successful operation occurred. This means that a thread will hop iff \textit{N^{active}{i}}=0pt for a Put operation, whereas for a Get operation, the thread will hop iff \textit{N^{active}{i}}=0. Therefore, the number of sub-structures, whose active regions are full, is given by \lfloor(\textit{N^{active}}/0pt)\rfloor at a given time, because the thread does not leave a sub-structure until its active region gets either full or empty. If the thread has to hop a sub-structure, then a new sub-structure is selected uniformly at random from the remaining set of sub-structures. If none of the sub-structures fulfills the condition (implies that \textit{N^{active}}=0 for a Get, or, \textit{N^{active}}=0pt\times 0pt for a Put), then the Window shifts based on a given shift parameter (i.e. for a Put operation \textit{Win_{max}}=\textit{Win_{max}}+\textit{shift^{up}} and for a Get operation \textit{Win_{max}}=\textit{Win_{max}}-\textit{shift_{down}}, where 1\leq\textit{shift_{down}},\textit{shift^{up}}\leq 0pt). One can observe that the value of $N^{active}$ before an operation defines the expected number of hops and the shift of the Window.

To compute the expected step complexity of an operation that occurs at a random time, we model the random variation process around the $Win_{max}$ with a Markov chain, where the sequence of Put and Get operations lead to the state transitions. We denote the Window shift parameter as $shift$, where shift=\textit{shift^{up}}=\textit{shift_{down}}=0pt. As a remark, we consider the performance of the sub-structures mostly when they are non-empty, since Get ($NULL$) and Put would have no hops in this case. The Markov chain is strongly related to $N^{active}$. It is composed of \textit{K}+1 states \mathcal{S}_{0},\mathcal{S}_{1},\dots,\mathcal{S}_{\textit{K}}, where \textit{K}=0pt\times 0pt. For all i\in\left\llbracket 0,\textit{K}\right\rrbracket, the operation is in state $\mathcal{S}_{i}$ iff \textit{N^{active}}=i. For all (i,j)\in\left\llbracket 0,\textit{K}+1\right\rrbracket^{2}, $\mathbb{P}\left(\mathcal{S}_{i}\rightarrow\mathcal{S}_{j}\right)$ denotes the state transition probability, that is given by the following function, where $p$ denotes the probability of a Put:

[TABLE]

The stationary distribution (denoted by the vector \pi=(\pi_{i})_{i\in\left\llbracket 0,\textit{K}\right\rrbracket}) exists for the Markov chain , since the chain is irreducible and positive recurrent (note that state space is finite). The left eigenvector of the transition matrix with eigenvalue $1$ provides the unique stationary distribution.

Lemma 1.

For the Markov chain that is initialized with $p=1/2$ and $shift$, where $l=shift\times 0pt-1$, the stationary distribution is given by the vector \pi^{l}=(\pi^{l}_{0}\pi^{l}_{1}..\pi^{l}_{\textit{K}}), assuming \textit{K}-l>=l (for $l>K-l$, one can obtain the vector from the symmetry \pi^{l}=\pi^{\textit{K}-l}): (i) \pi^{l}_{i}=\frac{i+1}{(l+1)(\textit{K}+1-l)},\text{ if }i<l; (ii) \pi^{l}_{i}=\frac{l+1}{(l+1)(\textit{K}+1-l)},\text{ if }l\leq i\leq\textit{K}-l; (iii) \pi^{l}_{i}=\frac{\textit{K}-i+1}{(l+1)(\textit{K}+1-l)},\text{ if }i>\textit{K}-l.

Proof.

We have stated that the stationary distribution exist since the chain is aperiodic and irreducible for all $p$ and $shift$.

Let (M_{i,j})_{(i,j)\in\left\llbracket 0,\textit{K}\right\rrbracket^{2}} denote the transition matrix for $p=1/2$ and $shift$. The stationary distribution vector $\pi^{l}$ fulfills, $\pi^{l}M=\pi^{l}$, that provides the following system of linear equations: (i) $2\pi^{l}_{0}=\pi^{l}_{1}$; (ii) 2\pi^{l}_{\textit{K}}=\pi^{l}_{(\textit{K}-1)}; (iii) $2\pi^{l}_{i}=\pi^{l}_{i-1}+\pi^{l}_{i+1}$; (iv) $2\pi^{l}_{l}=\pi^{l}_{i-1}+\pi^{l}_{i+1}+\pi^{l}_{1}$; (v) 2\pi^{l}_{\textit{K}-l}=\pi^{l}_{i-1}+\pi^{l}_{i+1}+\pi^{l}_{\textit{K}}.

In case, l=\textit{K}-l, then (iv) and (v) are replaced with 2\pi^{l}_{(l=\textit{K}-l)}=\pi^{l}_{i-1}+\pi^{l}_{i+1}+\pi^{l}_{1}+\pi^{l}_{\textit{K}}.

Based on a symmetry argument, one can observe that, for all $l$, \pi^{l}_{i}=\pi^{l}_{\textit{K}-i} the system can be solved in linear time (O(\textit{K})) by assigning any positive (for irreducible chain $\pi^{l}_{i}>0$) value to $\pi^{l}_{0}$. The stationary distribution is unique thus for any $\pi^{l}_{0}$, $\pi^{l}$ spans the solution space. We know that $\sum_{i=0}^{K}\pi^{l}_{i}=1$, starting from $\pi^{l}_{0}=1$, we obtain and normalize each item by the sum. ∎

An operation starts with the search of an available sub-structure. This search contains at least a single query at the sub-structure where the thread’s last successful operation occurred, the rest incur a hop step. In addition, the operation might include the shift of the window, as an extra step, denoted by $W_{shift}$. We denote the number of extra steps with Extra=hop+\textit{W_{shift}}. With the linearity of expectation, we obtain \mathbb{E}\left(Extra\right)=\mathbb{E}\left(hop\right)+\mathbb{E}\left(\textit{W_{shift}}\right). Relying on the law of total expectation, we obtain:

(i) \mathbb{E}\left(hop\right)=\sum_{i=0}^{\textit{K}}\sum_{\textit{op}\in\{pop,push\}}\mathbb{E}\left(hop|\mathcal{S}_{i},\textit{op}\right)\mathbb{P}\left(\mathcal{S}_{i},\textit{op}\right);

(ii) \mathbb{E}\left(\textit{W_{shift}}\right)=\sum_{i=0}^{\textit{K}}\sum_{\textit{op}\in\{pop,push\}}\mathbb{E}\left(\textit{W_{shift}}|\mathcal{S}_{i},\textit{op}\right)\mathbb{P}\left(\mathcal{S}_{i},\textit{op}\right);

where \mathbb{P}\left(\mathcal{S}_{i},\textit{op}\right) denotes the probability of an operation to occur in state $\mathcal{S}_{i}$. We analyze the algorithm for the setting where $shift=0pt$ and $p=1/2$. We do this because the bound, that we manage to find in this case, is tighter, and gives a better idea of the influence of the $2D$ parameters to the expected performance. For this case the stationary distribution is given by Lemma 1.

Theorem 2.

For a 2Dc-structure that is initialized with parameters 0pt, 0pt, $shift=0pt$ and $p=1/2$, $\mathbb{E}\left(Extra\right)=O(\frac{\ln 0pt}{0pt})$.

Proof.

Firstly, we consider the expected number of extra steps for a Put operation. Given that there are $N^{active}$ items, a Put attempt would generate an extra step if it attempts to add and item to a sub-structure that has \textit{N^{active}{i}}=0pt items. Recall that the thread sticks to a sub-structure until it is not possible to conduct an operation on it. This implies that the extra steps can be taken only in the states $\mathcal{S}_{i}$ such that $i(mod\text{ }0pt)=0$, because the thread does not leave a sub-structure before \textit{N^{active}{i}}=0 or \textit{N^{active}_{i}}=0pt. In addition, a Put (Get) can only experience an extra step if the previous operation was also a Put (Get).

Given that we are in $\mathcal{S}_{i}$ such that $i(mod\text{ }0pt)=0$, then the first requirement is to have a Put as the previous operation. If this is true, then the Put operation has to hop to another sub-structure, which is selected from the remaining set of sub-structures uniformly at random. At this point, there are $f=\frac{i}{0pt}-1$ full sub-structures in the remaining set of sub-structures. If a full sub-structure is selected from this set, this leads to another hop and again a sub-structure is selected uniformly at random from the remaining set of sub-structures.

Consider a full sub-structure (one of the $f$), this sub-structure would be hopped if it is queried before querying the sub-structures that are empty. There are $0pt-f-1$ empty sub-structures, thus a hop in this sub-structure would occur with probability $1/(0pt-f)$. There are $f$ such sub-structures. With the linearity of expectation, the expected number of hops is given by: $f/(0pt-f)+1=0pt/(0pt-f)$. Which leads to $\mathbb{E}\left(hop|\mathcal{S}_{i},\textit{Put}\right)=p\times 0pt/(0pt-f)$ if $i(mod\text{ }0pt)=0$ or $\mathbb{E}\left(hop|\mathcal{S}_{i},\textit{Put}\right)=0$ otherwise.

From Lemma 1, \pi_{i}<2/(\textit{K}+1) we obtain:

[TABLE]

The bounds for $\mathbb{E}\left(hop|\textit{Put}\right)$ also hold for $\mathbb{E}\left(hop|\textit{Get}\right)$. Given that there are \textit{K}-i (system is in state $\mathcal{S}_{i}$) empty sub-structures then there are e=\lfloor\frac{\textit{K}-i}{0pt}\rfloor-1 sub-structures whose Window regions are empty, minus the sub-structure that the thread last succeeded on. Using the same arguments that are illustrated above (replace $f$ with $e$ and p=1-p), we obtain the same bound.

Window only shifts at \mathcal{S}_{\textit{K}} if a Put operation happens and at $\mathcal{S}_{0}$ if a Get operation happens. Hence: \mathbb{E}\left(\textit{W_{shift}}\right)<\frac{2}{\textit{K}+1}p+\frac{2}{\textit{K}+1}(1-p). Finally, using \mathbb{E}\left(Extra\right)=\mathbb{E}\left(hop\right)+\mathbb{E}\left(\textit{W_{shift}}\right) we obtain the theorem. ∎

5.2. WinDecoupled

Now, we apply the same reasoning for the WinCoupled to analyse WinDecoupled behaviour. There are two Windows, let $Win_{max}^{put}$ and $Win_{max}^{get}$ represent $Win_{max}$ for Put and Get respectively. The two counters; $Win_{max}^{put}$ and $Win_{max}^{get}$, increase monotonically. Put and Get have the same complexity since they apply the same Window strategy with the only difference that one consumes and the other produces items. Therefore, we analyse only Put. Let $N^{active}_{i}$ denote the number of items in the active region of the sub-structure $i$ for Put. We consider $shift=0pt$ since the $Win_{max}^{put}$ is monotonically increasing.

We again model the process with a Markov chain where the states are strongly related to \textit{N^{active}}=\sum_{i=1}^{0pt}\textit{N^{active}_{i}}. It is composed of $K$ states \mathcal{S}_{1},\dots,\mathcal{S}_{\textit{K}}, where \textit{K}=0pt\times 0pt. For all (i,j)\in\left\llbracket 0,\textit{K}\right\rrbracket^{2}, $\mathbb{P}\left(\mathcal{S}_{i}\rightarrow\mathcal{S}_{j}\right)$ denotes the state transition probability, that is given by the following function: \mathbb{P}\left(\mathcal{S}_{i}\rightarrow\mathcal{S}_{i+1}\right)=1,\text{if }1\leq i\leq\textit{K}-1 and \mathbb{P}\left(\mathcal{S}_{i}\rightarrow\mathcal{S}_{1}\right)=1,\text{if }i=\textit{K}. The stationary distribution is given by the vector \pi^{l}=(\pi^{l}_{1}\pi^{l}_{2}..\pi^{l}_{\textit{K}}), where \pi_{i}=1/\textit{K}.

Theorem 3.

For a 2Dd-structure that is initialised with parameters 0pt, 0pt, $shift=0pt$, $\mathbb{E}\left(Extra\right)=O(\frac{\ln 0pt}{0pt})$.

Proof.

We consider the expected number of extra steps for an Put that would generate an extra step if it attempts on a sub-structure that has \textit{N^{active}_{i}}=0pt items. Recall that the thread sticks to a sub-structure until it is not possible to conduct an operation on it, thus extra steps will be taken only in the states $\mathcal{S}_{i}$ such that $i(mod\text{ }0pt)=0$ and before the first hop, there are $f=\frac{i}{0pt}-1$ full sub-structures in the remaining set of sub-structures. Plugging, \pi_{i}=1/(\textit{K}) into the reasoning that is provided in Theorem 2, we obtain the theorem. ∎

6. Deriving 2D Data structures

In this Section, we show how our framework can be used to derive k-out-of-order data structures. Using WinCoupled we derive a 2Dc-Stack and a 2Dc-Counter, whereas by using WinDecoupled, we derive a 2Dd-Stack, a 2Dd-Queue, a 2Dd-Counter and a 2Dd-Deque as shown in Table 1. The base algorithms include but not limited to; Treiber’s stack (Treiber1986systems, ), MS-queue (Michael:1996:SFP:248052.248106, ) and Maged Deque (DBLP:conf/europar/Michael03, ) for Stack, FIFO Queue and Deque respectively. In order to be precise and avoid reciting the base algorithms’ information, we focus our discussion on the algorithmic modifications made to fit our design framework.

6.1. 2D-Stack

A stack is characterised by two operations: Push that adds an item and Pop that removes an item from the stack. Our derived 2D-Stack algorithms are composed of multiple lock-free sub-stacks. Each sub-stack is implemented using a linked-list following the Treiber’s stack design (Treiber1986systems, ), modified only to fit the Window framework design as discussed below.

In Algorithm 3 we present the algorithmic implementation of 2Dc-Stack as an example of how to use WinDecoupled to derive k-out-of-order data structures. The stack head is modified to a descriptor containing the top item pointer, operation count, and descriptor version (Line 3). Note that, the descriptor is updated in single atomic step using a wide CAS (Line 3, 3), the same way as in the Treiber’s stack. 2Dc-Stack is accessed through a single array of descriptors for both Push and Pop operations.

To perform an operation, a given thread obtains a sub-stack by performing a Window search (Line 3, 3) as discussed earlier in Section 3. The thread then prepares a new descriptor based on the current descriptor at the given index (Line 3, 3). Using a CAS, the thread tries to atomically swap the current descriptor with the new one (Line 3, 3). If the CAS fails, the thread sets the contention indicator to true (Line 3, 3) and restart the Window search. The contention indicator signals the presence of contention prompting the Window search to starts from an index that is selected uniformly at random (Line 1) as discussed earlier in Section 3.

A successful Push increments whereas a Push decrements the operation count by one (Line 3, 3). Also, the topmost item pointer is updated. At this point, a Push adds an item whereas a Pop returns an item for a non-empty or $NULL$ for an empty stack (Line 3). An empty sub-stack is represented by a $NULL$ item pointer within the descriptor (Line 3). Recall that a thread performs a special data structure emptiness check, by scanning all the available sub-stacks and returning $NULL$ only if all the sub-stacks where found to be empty.

WinDecoupled is used to derive the 2Dd-Stack. This follows the same procedure as discussed above, with the difference being that; Push and Pop operations increment different operation counters on success. Also, we declare a single array of descriptors through which threads performing a Push operation or Pop can access any of the available sub-stacks. Each descriptor maintains two different counters; one for Push and the other for Pop. However, the Push and Pop operation Windows are independent because they follow different operation counts. Note that, we do not need version number because the counters increase monotonically.

6.2. 2D-Queue

FIFO Queues are characterised by two operations, Enqueue which adds an item to the queue and Dequeue which removes an item. The two operations access the queue from different points; head for Dequeue and tail for Enqueue. We use WinDecoupled to derive a 2D-Queue, due to its ability to maintain the independent operation counts and decriptor’s access arrays. 2D-Queue is composed of multiple lock-free sub-queues. Each sub-queue is implemented using a linked list following the Michael Scott FIFO queue (MS-queue) design (Michael:1996:SFP:248052.248106, ), modified only to fit the Window processes as shown in Algorithm 4.

The queue head and tail are modified into an independent descriptor (Line 4) for each. The head descriptor contains the head item pointer and the Dequeue operation counter, whereas the tail descriptor contains the tail item pointer and the Enqueue operation counter. The 2D-Queue descriptor is also updated in one atomic step using a wide CAS (Line 4, 4), same as in MS-queue updates. 2D-Queue composed of two arrays of descriptors; one holds the head descriptors (getDes), while the other holds the tail descriptors (putDes). Each sub-queue descriptor pair is uniquely identified by the same index value in both arrays.

To perform an operation, a given thread obtains a sub-queue by performing a Window search on a given descriptor array (Line 4, 4) as discussed earlier in Section 3. An Enqueue completes in two steps: First, the thread tries to add the new item to the queue list (Line 4), if successful, the thread then tries to update the tail descriptor with the new state (Line 4). If a thread encounters an incomplete Enqueue, the thread can help complete the pending Enqueue by updating the tail descriptor accordingly (Line 4, 4). CAS failure during an Enqueue or Dequeue operation signals the presence of contention (Line 4, 4, 4, 4, 4) on the given sub-queue. When contention is detected, the thread starts the next Window search on an index that is selected uniformly at random (Line 2) as discussed earlier in Section 3.

Both Enqueue and Dequeue increment their respective sub-queue descriptor operation count by one on success (Line 4, 4). Note that, Dequeue increments the Enqueue operation count if it helps complete a pending Enqueue (Line 4). On a successful operation, an Enqueue adds an item whereas a Dequeue returns a value for a non empty sub-queue (Line 4) or $NULL$ (Line 4) for an empty Queue. Similar to 2D-Stack, the Dequeue only returns $NULL$ if the search within a given Window cannot find a non-empty sub-queue after scanning all the available sub-queues.

6.3. 2D-Deque

Deques are characterised by four operations, PushLeft which adds an item to the left of the deque, PushRight which adds an item to the right of the deque, PopLeft which removes an item from the left of the deque, if any and PopRight which removes an item from the right of the deque, if any. We use WinDecoupled to derive a 2D-Deque, due to its ability have multiple Windows. Each of the four 2D-Deque operations is assigned an independent operation Window. 2D-Deque is composed of multiple lock-free sub-deques. Each sub-deque is implemented using a doubly-linked list following the Maged (Deque-Maged) design (DBLP:conf/europar/Michael03, ), modified only to fit the framework Window design as shown in Algorithm 5.

The sub-deque is modified, replacing the anchor with a descriptor that can also be updated in one atomic step using a CAS instruction, same as in Deque-Maged updates. The descriptor, holds two pointers to the leftmost and rightmost items in the sub-deque, if any, and a status tag. Four operation counters are added to the descriptor, one for each deque operation (Line 5). 2D-Deque is composed of a single array of descriptors through which the threads performing either of the four 2D-Deque operations can access any of the available sub-deques. We use a single array because our base algorithm Deque-Maged uses a single access point anchor for the four deque operations. For each operation on the given side of a sub-deque, a respective operation counter is incremented by one.

To perform an operation, a thread obtains a sub-deque by performing a Window search respective to the operation being performed (Line 5, 5, 5, 5) as discussed earlier in Section 3. A Push operation on either side is completed in one atomic step for an empty sub-deque by replacing the sub-deque descriptor with a new descriptor pointing to the new item (Line 5, 5). If the sub-deque is stable and contains one or more items, the Push operations complete in two steps. First step is to swing the given sub-deque descriptor pointer to the new item and to indicate unstable sub-deque in the status tag, atomically (Line 5, 5). After this step, the given sub-deque is unstable. The next step of a Push operation is to stabilise the sub-deque (Line 5, 5). A Pop operation on either side of a non empty sub-deque completes in one atomic step (Line 5, 5, 5, 5). For each operation, if the sub-deque is unstable, it must be stabilised (Line 5, 5, 5, 5), before attempting the given operation on the sub-deque.

CAS failure during any of the four operations signals the presence of contention (Line 5, 5, 5, 5, 5, 5, 5) on the given sub-deque. The same applies when a Dequeue operation on either sides of the sub-deque encounters an empty sub-deque (Line 5, Line 5). When contention is detected, the thread starts the next Window search on an index that is selected uniformly at random (Line 2) as discussed earlier in Section 3.

6.4. 2D-Counter

2D-Counter is characterized by two operations; increment (Put) which increases the counter and decrement (Get) which decreases the counter. It is composed of multiple sub-counters whose local count (sub-count) can only be greater than or equal to zero. Both 2Dc-Counter and 2Dd-Counter follow the 2Dc-Stack and 2Dd-Stack implementation details. Following the same strategy, a given successful operation, increments or decrements a given sub-counter’s sub-count, then calculates the estimated global count value as $Count=\textit{sub-count}\times 0pt$. $Count$ is then returned by the thread.

7. Correctness

In this section, we prove the correctness of the data structures we derived in Section 6, including their relaxation bounds and lock freedom. All our derived 2D data structures are linearizable with respect to k-out-of-order semantics for the respective data structure.

7.1. 2Dc-Stack

2Dc-Stack is linearizable with respect to the sequential semantics of k-out-of-order stack (henzinger2013quantitative, ). 2Dc-Stack Push and Pop linearization points are similar to those of the original Treiber’s Stack (Treiber1986systems, ). As shown in Algorithm 3, Pop linearizes either by returning NULL (Line 3) or with a successful CAS (Line 3). Push linearizes with a successful CAS (Line 3).

Relaxation can be applied method-wise and it is applied only to Pop operations, that is, a Pop pops one of the topmost $k$ items. Firstly, we require some notation. The Window defines the number of operations allowed to proceed on any given sub-stack. The Window is shifted by the parameter $shift,\ 1\leq shift\leq 0pt$ and $0pt=\#\textit{sub-stacks}$. For simplicity, let shift=\textit{shift_{up}}=\textit{shift_{down}}. A Window $i$ ( $W_{i}$) has an upper bound ( $W_{i}^{max}$) and a lower bound ($W_{i}^{min}$), where $W_{i}^{max}=i\times shift$ and $W_{i}^{min}=(i\times shift)-0pt$, respectively. For simplicity, let $Win_{max}$ represent the current global upper bound. A Window is active iff W_{i}^{max}=\textit{Win_{max}}. The number of items of the sub-stack $j$ is denoted by $N_{j}$, $1\leq j\leq 0pt$. To recall, the top pointer, the version number and $N_{j}$ are embedded into the descriptor of sub-stack $j$ and all can be modified atomically with a wide CAS instruction.

Lemma 1.

Given that \textit{Win_{max}}=shift\times i, it is impossible to observe a state($S$) such that $N_{j}>W_{i+1}^{max}$ (or $N_{j}<W_{i-1}^{min}$).

Proof.

We show that this is impossible by considering the interleaving of operations. Without loss of generality, assume thread $1$ ($P_{1}$) has set \textit{Win_{max}}=shift\times i at time $t_{1}\prime$. To do this, $P_{1}$ should have observed either \textit{Win_{max}}=shift\times(i-1) and then $N_{j}=W_{i-1}^{max}$ or \textit{Win_{max}}=shift\times(i+1) and then $N_{j}=W_{i+1}^{min}$. Let this observation of $Win_{max}$ happen at time $t_{1}$. Consider the last successful push operation at sub-stack $j$ before the state $S$ is observed for the first time (we do not consider Pop operations as they can only decrease $N_{j}$ to a value that is less than $W_{i+1}^{max}$, this case will be covered by the first item below). Assume thread [math] ($P_{0}$) sets $N_{j}$ to $N_{j}>W_{i+1}^{max}$ in this push operation. $P_{0}$ should observe $N_{j}\geq W_{i+1}^{max}$ and \textit{Win_{max}}>W_{i+1}^{max}. Let $j$ be selected at time $t_{0}$. And the linearization of the operation happens at $t_{0}\prime>t_{0}$.

• •

If $t_{0}\prime<t_{1}$, the concerned state($S$) can not be observed since $Win_{max}$ cannot be changed (to $shift\times i$) after $N_{j}>W_{i+1}^{max}$ is observed.

• •

Else if $t_{1}\prime<t_{0}$, the concerned state($S$) cannot be observed since the push operation cannot proceed after observing $Win_{max}$ with such $N_{j}$.

• •

Else if $t_{1}>t_{0}$, then $P_{0}$ cannot linearize because, this implies $N_{j}$ has been modified (the difference between the value of $Global$ that is observed by $P_{0}$ and then by $P_{1}$ implies this) since $P_{0}$ had read the descriptor, at least the version numbers would have changed since then.

• •

Else if $t_{1}<t_{0}$, then this implies $Win_{max}$ has been modified, since it was read by $P_{1}$, thus updating $Win_{max}$ would fail, at least based on the version number.

∎

Lemma 2.

At all times, there exist an $i$ such that $\forall j,1\leq j\leq 0pt$: $W_{i}^{min}\leq N_{j}\leq W_{i+1}^{max}$.

Proof.

Informally, the lemma states that the size (number of operations) of a sub-stack spans to at most two consecutive accessible Windows. Assume that the statement is not true, then there should exist a pair of sub-stacks ($y$ and $z$) at some point in time such that $\exists i,N_{y}<W_{i}^{min}$ and $N_{z}>W_{i+1}^{max}$. Consider the last Push at sub-stack $z$ and last Pop at sub-stack $y$ that linearize before or at the time $t$.

Assume thread $P_{0}$ (Push) sets $N_{z}$ and thread $P_{1}$ (Pop) sets $N_{y}$. To do this, $P_{0}$ should observe $N_{z}\geq W_{i+1}^{max}$ and \textit{Win_{max}}>W_{i+1}^{max}, let sub-stack $z$ be selected at $t_{0}$. And, the linearization of the Push operation occurs at $t_{0}\prime>t_{0}$. Similarly, for $P_{1}$ Pop operation, let sub-stack $y$ be selected at $t_{1}$, $P_{1}$ should have observed \textit{Win_{max}}\leq W_{i}^{min}. And, let the Pop operation linearize at time $t_{1}\prime>t_{1}$. Now, we consider the possible interleavings.

• •

If $t_{0}\prime<t_{1}$ (or the symmetric $t_{1}\prime<t_{0}$ for which we do not repeat the arguments), then for $P_{1}$ to proceed and pop an item from sub-stack $y$, it is required that \textit{Win_{max}}\leq W_{i}^{min}. Based on Lemma 1, this is impossible when $N_{z}>W_{i}^{max}$.

• •

Else if $t_{1}>t_{0}$, then $P_{0}$ cannot linearize, because this implies that $N_{z}$ has been modified (the difference between the value of $Win_{max}$ that is observed by $P_{0}$ and then by $P_{1}$ implies this) since $P_{0}$ has read the descriptor. At least, the version number would have changed since then.

• •

Else if $t_{0}>t_{1}$, the argument above holds for $P_{1}$ too, so $P_{1}$ should fail to linearize.

Such $N_{z}$ and $N_{y}$ pair can not co-exist at any time. ∎

Theorem 3.

2Dc-Stack is linearizable with respect to k-out-of-order stack semantics, where $k=(2shift+0pt)(0pt-1)$.

Proof.

Consider the Push ($t_{e}^{push}$) and Pop ($t_{e}^{pop}$) linearization points, that insert and remove an item $e$ for a given sub-stack $j$ respectively, where, $t_{e}^{pop}>t_{e}^{push}$. Now, we bound the maximum number of items, that are pushed after $t_{e}^{push}$ and are not popped before $t_{e}^{pop}$, to obtain $k$. Let item $e$ be the ${N_{j}}^{th}$ item from the bottom of the sub-stack. Consider a Window $i$ such that: $W_{i}^{min}\leq N_{j}\leq W_{i}^{max}$.

Lemma 2 states that the sizes of the sub-stacks should reside in a bounded region. Relying on Lemma 2, we can deduce that at time $t_{e}^{push}$, the following holds: $\forall i:N_{j}\geq W_{i}^{min}-shift$. Similarly, we can deduce that at time $t_{e}^{pop}$, the following holds: $\forall i:N_{j}\leq W_{i}^{max}+shift$. Therefore, the maximum number of items, that are pushed to sub-stack $j$ after $t_{e}^{push}$ and are not popped before $t_{e}^{pop}$ is at most $W_{i}^{max}+shift-(W_{i}^{min}-shift)=0pt+2shift$. We know that this number is zero for sub-stack $j$ (the sub-stack that $e$ is inserted) and we have $0pt-1$ other sub-stacks. So, there can be at most $(2shift+0pt)(0pt-1)$ items that are pushed after $t_{e}^{push}$ and are not popped before $t_{e}^{pop}$. ∎

7.2. 2Dd-Stack

Theorem 4.

2Dd-Stack is linearizable with respect to k-out-of-order stack semantics, where $k=(30pt)(0pt-1)$.

Proof.

Consider the linearization points of Push and Pop operations that respectively insert and remove the item $e$ into and from a sub-stack (sub-stack $i$). Let $t_{e}^{push}$ and $t_{e}^{pop}$ denote these points, respectively. Now, we bound the maximum number of items, that are pushed after $t_{e}^{push}$ and are not popped before $t_{e}^{pop}$, to obtain $k$. We denote the number items that are pushed to (popped from) sub-stack $j$ in the time interval $[t_{e}^{push},t_{e}^{pop}]$, with ${push}_{j}$ (${pop}_{j}$).

Regarding the interval $[t_{e}^{push},t_{e}^{pop}]$, we have: (i) ${push}_{i}={pop}_{i}$, since the number of items that are pushed into and popped from sub-structure $i$ should be equal; (ii) $\forall j\in[1,0pt],push_{j}\leq push_{i}+0pt+(0pt-(push_{i}\mod 0pt))$; and (iii) $\forall j\in[1,0pt],pop_{j}\geq pop_{i}-0pt-(pop_{i}\mod 0pt)$.

Therefore, for any sub-stack, the number of items that are pushed after $t_{e}^{push}$ and are not popped before $t_{e}^{pop}$ ($push_{j}-pop_{j}$) is at most: $push_{i}+0pt+(0pt-(push_{i}\mod 0pt))-(pop_{i}-0pt-(pop_{i}\mod 0pt))=30pt$. Summing over all $j\neq i$, we obtain the theorem. ∎

7.3. 2Dd-Queue

Theorem 5.

2Dd-Queue is linearizable with respect to k-out-of-order stack semantics, where $k=(0pt)(0pt-1)$.

Proof.

The linearization points for 2Dd-Queue operations follow MS-queue design (Michael:1996:SFP:248052.248106, ). As shown in Algorithm 4, Dequeue linearizes either by returning NULL (Line 4) or with a successful CAS (Line 4). Push linearizes with a successful CAS (Line 4). For readability reasons, We use $Win_{max}^{put}$ and $Win_{max}^{get}$ as a representative for Enqueue and Dequeue $Win_{max}$ respectively. For Enqueue (Dequeue), 2Dd-Queue algorithm searches for a sub-queue whose enqueue (dequeue) counter is smaller than $Win_{max}^{put}$ ($Win_{max}^{get}$). If no such sub-queue exist, the $Win_{max}^{put}$ ($Win_{max}^{get}$) is increased by $0pt$.

One can observe that $Win_{max}^{put}$ and $Win_{max}^{get}$ are monotonically increasing. Also, \textit{Win_{max}^{put}}=i\times 0pt iff the Enqueue counter for all sub-queues are in the range $[(i-1)\times 0pt+1,i\times 0pt]$. The same holds for $Win_{max}^{get}$. \textit{Win_{max}^{put}}^{i} is updated to \textit{Win_{max}^{put}}^{i+1}=\textit{Win_{max}^{put}}^{i}+0pt as a result of Enqueue iff Enqueue counter for all sub-queues are equal to \textit{Win_{max}^{put}}^{i} before the Enqueue. These invariants hold at all times since the concurrent operations, that might violate them, would fail at CAS instructions that modifies any sub-queue or any $Win_{max}$.

Based on these invariants, all the items that are enqueued while \textit{Win_{max}^{put}}=i\times 0pt will be dequeued while \textit{Win_{max}^{get}}=i\times 0pt. Therefore, the maximum number of items that are enqueued before an item and are not dequeued before that item can be at most $0pt\times 0pt$. Disregarding the items that are enqueued on the same sub-queue, we obtain the theorem. ∎

7.4. 2Dd-Deque

Theorem 6.

2Dd-Deque is linearizable with respect to k-out-of-order dequeue semantics, where $k=(80pt)(0pt-1)$.

Proof.

2D-Deque Push and Pop linearization points are similar to those of the original Maged Deque (DBLP:conf/europar/Michael03, ). As shown in Algorithm 5, Pop linearizes either by returning NULL (Line 5, 5) or with a successful CAS (Line 5, 5, 5, 5). Push linearizes with a successful CAS (Line 5, 5, 5, 5).

Consider the linearization points of Push and Pop operations that respectively insert and remove the item $e$ into and from a sub-deque $i$. Let $t_{e}^{push}$ and $t_{e}^{pop}$ denote these points, respectively.

First, we consider the case where Push and Pop operations on item $e$ occur at the same side of the sub-deque. Without loss generality, we assume they happen at the right side of the sub-deque. In this case, we bound the maximum number of items, that are pushed after $t_{e}^{push}$ from right and are not popped before $t_{e}^{pop}$, to obtain $k$. We denote the number items that are right pushed to (popped from) sub-deque $j$ in the time interval $[t_{e}^{push},t_{e}^{pop}]$, with ${push}_{j}$ (${pop}_{j}$). Regarding the interval $[t_{e}^{push},t_{e}^{pop}]$, we have: (i) ${push}_{i}={pop}_{i}$, since the number of items that are pushed into and popped from sub-deque $i$, from the same side, should be equal; (ii) $\forall j\in[1,0pt],push_{j}\leq push_{i}+20pt$; and (iii) $\forall j\in[1,0pt],pop_{j}\geq pop_{i}-20pt$. Summing over all $0pt$ sub-deques, we have for pushes at most: $\sum_{j=0}^{0pt-1}push_{j}\leq 0pt(push_{i})+2(0pt-1)0pt$ and for pops at least: $\sum_{j=0}^{0pt-1}pop_{j}\geq 0pt(pop_{i})-2(0pt-1)0pt$. Therefore, the number of items that are pushed from right after $t_{e}^{push}$ and are not popped before $t_{e}^{pop}$ ($push_{j}-pop_{j}$) is at most: $\sum_{j=0}^{0pt-1}push_{j}-\sum_{j=0}^{0pt-1}pop_{j}\leq 4(0pt-1)0pt$.

Second, we consider the case where Push and Pop operations on item $e$ occur at the opposite sides of the deque. Without loss generality, we assume Push operation happens at the right side of the deque. In this case, we bound the maximum number of items: (i) that are pushed after $t_{e}^{push}$ from left and are not popped before $t_{e}^{pop}$; (ii) the items that are already inside the deque at $t_{e}^{push}$ and are not popped before $t_{e}^{pop}$. Summing these two terms will provide an upper bound (though not necessarily a tight one) and we obtain $k$. Let $size_{j}$ denotes the size of the sub-deque $j$ (number of items inside deque $j$) at time $t_{e}^{push}$. $pushLeft_{j}$ and $popLeft_{j}$ denotes the number of item that are pushed (and popped) to (from) sub-deque $j$ from left in the time interval $[t_{e}^{push},t_{e}^{pop}]$. We know that for sub-deque $i$, we have $popLeft_{i}=pushLeft_{i}+size_{i}$. And, we have the following three relations: $\forall j\in[1,0pt],popLeft_{i}\leq popLeft_{j}+20pt,pushLeft_{i}\geq pushLeft_{i}-20pt,size_{i}\geq size_{j}-40pt$. Thus, we obtain $\forall j\in[1,0pt],80pt\geq pushLeft_{j}+size_{j}-popLeft_{j}$. Summing over all sub-deques $80pt(0pt-1)\geq\sum_{j=0}^{0pt-1}pushLeft_{j}+size_{j}-popLeft_{j}$, we obtain an upper bound for the maximum number of items that could be covered by (i) and (ii).

Finally, we obtain $k$ for a Pop operation by taking the maximum of case one and two:

$max(80pt(0pt-1),40pt(0pt-1))=80pt(0pt-1)$. ∎

7.5. 2Dc-Counter

Theorem 7.

2Dc-Counter is linearizable with respect to k-out-of-order counter semantics, where $k=(shift+0pt)(0pt-1)$.

Proof.

Lemma 2 states that the size of sub-structure can span to at most two consecutive accessible Windows, which implies that the difference between any two sub-counters can be at most $20pt$ at any point in time. Let $counter_{i}$ denotes the counter value for sub-counter $i$. One can observe an error at most $(shift+0pt)(0pt-1)$ because $\lVert\sum_{j=1}^{0pt}counter_{j}-(0pt)counter_{i}\rVert\leq(shift+0pt)(0pt-1)$. ∎

7.6. 2Dd-Counter

Theorem 8.

2Dd-Counter is linearizable with respect to k-out-of-order counter semantics, where $k=(20pt)(0pt-1)$.

Proof.

Let $counter_{i}^{decrement}$ (resp. $counter_{i}^{increment}$) denote decrement (resp. increment) counters for sub-counter $i$. We know that $\forall i,j\in[1,0pt]:\lVert(counter_{i}^{increment}-counter_{i}^{decrement})-(counter_{j}^{increment}-counter_{j}^{decrement})\rVert\leq 20pt$. Informally, the difference between the sizes of any two sub-counters can be at most $20pt$ at any point in time. The rest of the proof follows as Theorem 8. ∎

7.7. Lock-freedom

WinCoupled, Window shifting is lock-free iff $shift<0pt$ and obstruction free iff $shift=0pt$. Take an example, a Get operation might read an empty Window and shift it down to a full state, but before it selects a sub-structure, a subsequent Put reads the full Window state and shifts it up to an empty Window state. It is possible for this process continue forever leading to a system live lock. This is however avoided by setting the shift parameters to less than 0pt.Unlike WinCoupled, WinDecoupled is always lock-free.

Each sub-structure is lock-free: An operation can fail on CAS only if there is another successful operation. A Window shift can only fail if there is another successful shift operation preceded by a successful Put or Get, ensuring system progress. Thus, all our derived algorithms are lock-free with the exception of possible obstruction freedom as discussed above.

8. Other Algorithms for Comparison

To facilitate a detailed study, we implement other given relaxed data structure algorithms using three extra relaxation techniques following the same multi sub-structure design; Random, Random-C2 and Round-Robin. These present a combination of characteristics that add value to our evaluation, as shown in Table 2. Just like our design framework, we use the 0ptparameter to define the number of sub-structures for all the derived algorithms.

For Random, a Put or Get operation selects a sub-structure uniformly at random and proceeds to operate on the given sub-structure, whereas for Random-C2, a Get operation randomly selects two sub-structures, compares their items returning the most correct depending on the data structure semantics (rihani2015brief, ; Alistarh:2017:PCP:3087801.3087810, ). Under Random-C2, Put operations time stamp items marking their time of entry. It is these timestamps that are compared to determine the order among the two items during a Get operation. Due to the randomised distribution of operations, we expect low or no contention, low or no locality. These three characteristics help us compare and contrast with our optimisations discussed in Section 4. Using the Random design technique we derive a S-random stack, a Q-random queue, a C-random counter and a Deque-Random deque. Using the Random-C2 technique we derive a S-random-c2 stack, a Q-random-c2 queue and a C-random-c2 counter. We use similar base algorithms used to derive 2D algorithms discussed in Section 6 for each data structure. Random and Random-C2 derived algorithms do not provide deterministic k-out-of-order relaxation bounds.

Under Round-Robin, a thread selects and operates on a sub-structure in a strict round-robin order following the thread local counter. The thread must succeed on the selected sub-structure before proceeding to the next. This implies that in case of contention, a failing thread must retry on the same sub-structure until the given thread succeeds. Due to retries on contended sub-structures by the contending threads, we expect more contention to be generated from the retries. The thread always selects a different sub-structure after a successful operation, hence we expect low or no locality. However, memory access using round robin scheduling can take advantage of hardware prefecthing, a good characteristic to compare and contrast with our locality optimisation. Using Round-Robin technique, we derive a S-robin stack, a Q-robin queue, a C-robin counter and a Deque-robin. Round-Robin provides deterministic k-out-of-order relaxation bounds, we demonstrate this using S-robin whose bound is given by Theorem 1.

8.1. S-robin Correctness

Theorem 1.

S-robin is linearizable with respect to k-out-of-order stack semantics, where $k=(2P-1)(0pt-1)$. Where $P$ is number of threads and 0ptis the number of sub-stacks

Proof.

Consider the linearization points of Push and Pop operations that respectively insert and remove the item $e$ into and from a sub-stack (let sub-stack [math]). Let $t_{e}^{push}$ and $t_{e}^{pop}$ denote these points, respectively. Now, we bound the maximum number of items, that are pushed after $t_{e}^{push}$ and are not popped before $t_{e}^{pop}$, to obtain $k$. We denote the number items that are pushed to (popped from) sub-stack $i$ by thread $j$ in the time interval $[t_{e}^{push},t_{e}^{pop}]$, with ${push}_{i}^{j}$ (${pop}_{i}^{j}$).

Observe that each thread applies its operations in round robin fashion without skipping any sub-stack. If the previous successful Pop had occurred at sub-stack $i$, the next Pop occurs at sub-stack $i+1(mod\text{ }0pt)$. The same applies for the push operations.

Without loss of generality, assume that thread [math] has inserted item $e$ to sub-stack [math]. This implies that $\forall i,0pt-1\geq i>0,\ {push}_{0}^{0}\geq{push}_{i}^{0}$. Now, take another thread $j$, we have $\forall i:0pt-1\geq i>0,{push}_{0}^{j}\geq{push}_{i}^{j}-1$. Informally, another thread can increase the number of items on any other sub-stack by at most one more compared to the number of items that pushes on sub-stack [math].

For the pop operations, we have the same relation for all threads: $\forall i,0pt\geq i>0,\ {pop}_{0}^{j}\geq{pop}_{i}^{j}+1$. Informally, a thread can pop at most 1 item less from any other sub-stack compared to the number that it pops from sub-stack [math]. As the interval $[t_{e}^{push},t_{e}^{pop}]$ starts with the push and ends with the pop of item $e$ at sub-stack [math], we have $\sum_{j=0}^{P-1}{push}_{0}^{j}=\sum_{j=0}^{P-1}{pop}_{0}^{j}=Y$.

Summing over all threads and sub-stacks other than sub-stack [math], we get at most $(Y+P-1)(0pt-1)$ Push operations in the interval $[t_{e}^{push},t_{e}^{pop}]$. Summing over all threads and sub-stacks other than sub-stack [math], we get at least $(Y-P)(0pt-1)$ Pop operations. Which leads to the theorem: $k\leq((Y+P-1)-(Y-P))(0pt-1)=(2P-1)(0pt-1)$ ∎

9. Experimental Evaluation

We experimentally evaluate the performance of our derived algorithms, in comparison to k-out-of-order relaxed algorithms available in the literature, and other state of the art data structure algorithms. k-out-of-order relaxed algorithms include; Least recently used queue (lru) (Haas:2013:DQS:2482767.2482789, ), Segmented queue (Q-segment) and k-Stack (afek2010quasi, ; henzinger2013quantitative, ), other algorithms include; MS-queue (MS-queue) (Michael:1996:SFP:248052.248106, ), Wait free queue (wfqueue) (Yang:2016:WQF:2851141.2851168, ), general doubly linked list Deque (Deque-Sundell) (DBLP:journals/jpdc/SundellT08, ), Maged deque (Deque-Maged) (DBLP:conf/europar/Michael03, ), Time stamped stack (TS-Stack) (Dodds:2015:SCT:2775051.2676963, ) and Elimination back-off stack (Elimination) (hendler2010scalable, ). 0ptwill be generally used to refer to number of sub-structures for all algorithms using multiple sub-structures.

To facilitate a uniform comparison, we implemented all the evaluated algorithms using the same development tools and environment. The source code is publicly available on this link https://github.com/dcs-chalmers/Semantic-relaxation-2D-design-framework.

9.1. System Description

Experiments are run on two x86-64 machines: (i) Intel Xeon E5-2687W v2 dual socket machine (Multi-S) and (ii) Intel Xeon Phi 7290 single socket machine (Single-S). Multi-S has 8-core Intel Xeon processors per socket, each running at 3.4GHz, with three cache levels: core private L1 cache = 32 KB, core private L2 cache = 256 KB and a socket intra shared L3 cache = 25.6 MB. Single-S has 72-core processor running at 1.5GHz, core private L1 cache = 32 KB, L2 cache = 1024KB shared by two cores (tile). Multi-S and Single-S run on Ubuntu 16.04.2 LTS and CentOS Linux 7 Core Operating systems receptively. The Multi-S machine is used to evaluate inter-socket execution behaviour, whereas Single-S is used to evaluate intra-socket with high number of threads. Threads are pined one per core, for both machines excluding hyper-threading. Inter-socket execution is evaluated through pinning the threads one per core per socket in round robin fashion. Threads randomly select between Put or Get per operation with a given probability (operation rate). Memory is managed using the ASCYLIB framework SSMEM (David:2015:ACS:2786763.2694359, ).

Our main goal is to achieve scalability under high operation rate. To evaluate this, we simulate high operation rate by excluding work between operations. To reduce the effect of $NULL$ returns 222Usually $NULL$ returns are very fast because they do not update the data structure state, if not minimised, they can give misleading performance results for Get operations, all algorithms are initialised with $2^{17}$ items. Each experiment is then run for five seconds obtaining an average of five repeats. Throughput is measured in terms of operations per second, whereas the relaxation behaviour (accuracy) is measured in terms of the error distance from the exact data structure sequential semantics (henzinger2013quantitative, ). The higher the error distance, the lower the accuracy.

9.2. Measuring accuracy

We adopt a similar methodology used in the literature (alistarh2015spraylist, ; rihani2015brief, ). A sequential linked-list (doubly-linked list for deque) is run alongside the data structure being measured. Items on the data structure are duplicated on the linked-list and can be identified by their unique labels. For each operation Put or Get, a simultaneous insert or delete is performed on the linked-list respectively, following the exact semantics of the given data structure. A global lock is carefully placed at the data structure linearization points, locking both the linked-list and the data structure simultaneously. The lock allows only one thread to update both the data structure and the linked-list in isolation.

A given thread has to acquire the lock before it tries to linearize on any given sub-structure. Note that, Window search is independent of the lock. Put operations happen at the head or tail of the list for LIFO or FIFO measurements respectively (left or right for deque). A Get operation searches for the given item on the linked list, deletes it, and returns its distance from the respective access point, head or tail (error distance). For counter measurements, we replace the linked-list with a single fetch and add (FAA) counter. Both counters are updated in isolation using a global lock like explained above. The error distance is calculated from the difference between the two counter values.

Experiment results are then plotted using logarithmic scales, throughput (solid lines) and error distance (dotted lines) sharing the x-axis.

9.3. Dimension Tunability

Our design framework is tunable, giving designers the ability to manage performance optimizations for different execution environments and workloads, within a given tight relaxation bound ($k$). To evaluate this, we experiment with different 0ptand 0ptparameter configurations, as shown in Figures 3 and 3 for 2Dc-Stack (representative of WinCoupled) and 2Dd-Queue (representative of WinDecoupled) respectively. Curve (D1) depicts the case for fixed $0pt=1$ which also represents a case of relaxing in one dimension (increasing the number of sub-structures). The other curves (1P,2P,3P,4P,5P and 6P) depict execution in two dimensions with a set maximum $0pt\in\{1P,2P,3P,4P,5P,6P\}$, where $P$ is the number of threads. As relaxation increases, we increase 0ptuntil the maximum 0pt, then switch to increasing 0ptwithin the specified relaxation bound. For simplicity, 0ptis described as a multiple of the number of threads. It should however be noted that 0ptcan be configured to be independent of the number of threads.

D1 presents the lowest throughput as $k$ increases. This is attributed to the increasing 0ptproportional to $k$, leading to increased hops and lack of locality exploitation. On the other hand, we observe improved throughput performance for two dimensional executions. For all measured $k$, we observe that there is no consistent optimal 0ptconfiguration. This implies that, an optimal configuration is dependant on other factors, including; $k$ relaxation, type of workload, plus accuracy vs throughput trade-off. There are also notable differences between Single-S and Multi-S results. This calls for a multi-objective optimisation model, which is beyond the scope of this article.

With respect to the evaluated cases, we observe that $0pt=3P$ provides a fair balance between accuracy and throughput performance especially for 2Dc-Stack as shown in Figure 3. However for smaller $k$, there are varying high throughput points. Since we do not have an optimisation model, we empirically obtain the high throughput 0ptconfigurations for different executions.

However for Multi-S, we note that smaller 0pt($1P$) can achieve higher throughput performance. This is attributed to the high inter socket communication cost, having smaller 0ptallows for the exploitation of locality through relaxing more in the 0ptdimension. Exploiting locality reduces the communication between sockets, in turn, avoiding the inter socket communication cost.

9.4. Scaling With Relaxation

In order to evaluate monotonicity with increasing relaxation bound ($k$), we fix the number of threads to $16$. This is to match the number of cores available on Multi-S without hyper-threading. Results are presented in Figures 5, 7, 5 and 7 for stack, queue, counter and deque respectively.

First, we observe the difference between WinCoupled and WinDecoupled for 2Dc-Stack and 2Dd-Stack respectively in Figure 5. 2Dc-Stack consistently outperforms 2Dd-Stack due to the reduced Window shifting updates. With 2Dc-Stack, a given thread can locally operate on the same sub-stack longer since operation counts cancel out each other leaving the sub-stack in a valid state. The longer a given sub-stack stays valid, the higher the chances of exploiting locality. This advantage is more evident with symmetric workloads (50% push rate). As the workload becomes more asymmetric (70% push-rate), less Push counts are cancelled out by Pop counts. This implies that, the Window gets full more frequent leading to increased Window shifts. With 100% asymmetric workloads, 2Dc-Stack and 2Dd-Stack present similar execution behaviour. The same is observed for 2Dc-Counter and 2Dd-Counter in Figure 5.

All multi sub-structure based algorithms increase their 0pt(number of sub-structures) as $k$ increases to reduce contention and allow for increased disjoint access as shown in Table 2. However, for k-Stack, Q-segment, and lru, hops increase as 0ptincreases, this explains their observed low throughput gain. S-robin, Q-robin and C-robin are not affected by hops. However, for smaller $k$ values, they suffer from high contention arising from contending threads retrying on the same sub-structure until they succeed. As contention vanishes with high $k$ values, throughput gain saturates due to lack of locality. Round-Robin algorithms take advantage of the hardware prefetching available on both Single-S and Multi-S machines to reduce on the downside effect of lack or locality. This explains the observed throughput gain as 0ptincreases.

$2D$ algorithms maintain throughput gain through limiting 0ptto a size beneficial to reducing contention and switch to adjusting the 0ptto reduce hops. For our evaluation, $2D$ algorithms’ 0ptis configured as shown in Table 2. Once the algorithm attains the configured 0pt, it switches to increasing 0ptas $k$ increases. The 0ptparameter allows $2D$ algorithms to maintain throughput gain (monotonicity) through exploiting locality while reducing latency. This is observed for both Single-S and Multi-S machines.

In terms of accuracy, we observe an almost linear decrease in accuracy as $k$ increases for all algorithms. However, for 2Dc-Stack we observe a reduced rate of accuracy loss when the algorithm switches to increasing 0pt.

9.5. Scaling With Threads

To evaluate the scalability of our design as the number of threads increases, we fix the relaxation bound to ($k=10^{4}$) and vary the number of threads as shown in Figures 9, 11 and 9 for stack, queue and counter respectively. The reason for $k=10^{4}$ is to reduce the effect of contention due to small 0ptat lower $k$ values. This helps us focus on scalability effects. $2D$ algorithms’ 0ptis configured as shown in Table 2. Random and Random-C2 algorithms’ 0ptis set to $3P$, as the optimal balance between throughput and accuracy (don2dstackreport, ) since both of them do not provide a deterministic $k$ relaxation bound but rather a probabilistic one where applicable (Alistarh:2017:PCP:3087801.3087810, ; DBLP:conf/esa/Williams0D21, ).

k-Stack and Q-segment maintain a constant segment size as the number of threads increases. This increases the rate at which segments get filled up, leading to a high frequency of hops and segment maintenance cost especially for asymmetric workloads. As observed, throughput gain quickly saturates even for a lower number of threads leading to limited scalability.

The scalability of lru is mostly limited by the global counter used to calculate the last recently used sub-queue. For every operation, the thread has to increment the global counter using a FAA instruction, turning the counter into a scalability bottleneck. This can be observed when lru performance is compared to that of a single FAA counter (C-FAA). wfqueue suffers from the same FAA counter sequential bottleneck.

TS-Stack’s throughput is limited by the Pop search retries, searching for the newest item. Moreover, Pop operations might contend on the same newest items if there are not enough concurrent Push operations. Also, Pop lacks locality, which explains the drop in throughput on the Multi-S machine, due to the high inter-socket communication costs. We observe that throughput increases with increased Push rate. This is due to increased local processing and increased number of generated young items, leading to reduced thread contention for Pop operations.

For Round-Robin algorithms, the 0ptis inversely proportional to the number of threads (See Theorem 1). As the number of threads increases, 0ptreduces leading to increased contention. This explains the observed drop in throughput for a high number of threads, especially for the S-robin and the C-robin algorithms due to their sub-structure single access design. The effect of lack of locality can be reduced by hardware pre-fetching, a feature available on both machines. This can also explain the Round-Robin better performance compared to the performance of the other algorithms that lack locality.

Random and Random-C2 algorithms are affected by the lack of locality, which is evident by the difference between Single-S and Multi-S results. We observe that the performance difference between Random and $2D$ algorithms increases on the Multi-S machine as compared to that on the Single-S machine. This demonstrates how much $2D$ algorithms gain from exploiting locality when executing on a Multi-S machine. Locality helps to avoid paying the high inter-socket communication cost through improved caching behaviour (Hackenberg:2009:CCA:1669112.1669165, ; David:2013:EYA:2517349.2522714, ; Schweizer:2015:ECA:2923305.2923811, ; DBLP:conf/spaa/RukundoAT22, ).

10. Conclusion

In this work, we have shown that semantics relaxation has the potential to monotonically trade relaxed semantics of concurrent data structures for achieving throughput performance within tight relaxation bounds. This has been achieved through an efficient two-dimensional framework that is simple and easy to implement for different data structures. We demonstrated that, by deriving two-dimensional lock-free designs for stacks, FIFO queues, dequeues and shared counters.

Our experimental results have shown that relaxing in one dimension, restricts the capability to control relaxation behaviour in-terms of throughput and accuracy. Compared to previous solutions, our framework can be used to extend existing data structures with minimal modifications while achieving better performance in terms of throughput and accuracy.