Making data center computations fast, but not so furious

TL;DR

This paper introduces an aggressive computational sprinting approach for data centers that leverages existing replication to enhance performance beyond traditional safety limits.

Contribution

It proposes a novel sprinting variant that exploits data center replication, expanding beyond prior work focused on safety and non-faulty conditions.

Findings

Survey of existing sprinting techniques

Design ideas for future implementation

Potential for increased data center performance

Abstract

We propose an aggressive computational sprinting variant for data center environments. While most of previous work on computational sprinting focuses on maximizing the sprinting process while ensuring non-faulty conditions, we take advantage of the existing replication in data centers to push the system beyond its safety limits. In this paper we outline this vision, we survey existing techniques for achieving it, and we present some design ideas for future work in this area.

Full text

Computational Sprinting, Overclock, Fault tolerance, Data center.

††copyright: none††doi: 10.475/123_4††isbn: 123-4567-24-567/08/06††conference: The 7th Workshop on Multi-core and Rack Scale Systems; April 2017; Belgrade, Serbia††journalyear: 2017††price: 15.00

1. Introduction

Today’s global scale Internet services run in large data center infrastructures and are accessed by millions of users. The sheer scale in which these systems operate is such that the design of the infrastructure underlying data center systems must expect an environment where faults are the norm, and no longer the exception.

A key technique for building systems to provide high availability despite faults is to employ redundancy, often through the use of distributed replication protocols. However, this redundancy has a resource usage cost associated with it. For instance, Google uses 5 replicas for the F1 Advertising Back-end (Corbett et al., 2012), which multiplies the number of servers required to run this service by that factor.

Furthermore, F1 is not an isolated example, since redundancy is present in a large fraction of the systems that are part of the software stack of major companies such as Google or Facebook (Schwarzkopf, 2015). Thus, redundantly storing data and performing computations in multiple servers increases the energy cost and uses resources that otherwise could be allocated to serve other types of requests. Nevertheless, this expense is seen as an important insurance, because faults lead to service downtime which, in turn, affects revenue (DeCandia et al., 2007).

On an orthogonal direction, energy efficiency is also a concern for data center operators, as it impacts the requirements and consequently the cost of the infrastructure for power delivery. This is a pressing problem both because the power consumption of servers is increasing with the advances in density and number of cores, and because the cost associated with an increase in power capacity can be very high, reaching tens of thousands of USD per additional MegaWatt (Fan et al., 2007).

Moreover, while over-subscription of data centers’ power supply allows for accommodating infrequent correlated spikes in server power consumption, it exposes data centers to the risk of tripping power breakers and causing outages. For instance, Facebook initially had to disable dynamic overclocking (Intel Turbo Boost (Intel, 2011)) in one of its clusters, due to an insufficient power margin, despite the potential benefits in performance of this technology. To safely enable turbo mode while avoiding the risk of outages in high load periods, they designed a power management system to cap energy consumption according to the data center power budget  (Wu et al., 2016).

To lower energy consumption while also maintaining and even improving performance, a recent approach called computational sprinting (Raghavan et al., 2012) exploits the thermal capacitance of materials to activate cores (parallel sprinting) or overclocking the CPU (frequency sprinting), thus exceeding the sustained cooling capabilities of the system for short periods. As a result, this scheme is able to improve application responsiveness by up to 6x and save about 30% on power for a conventional Core i7 Desktop chip, as a consequence of finishing computations in a shorter amount of time (Raghavan et al., 2013). This technique was also extended to data centers, in which more interactive workloads such as search or news feeds, that exhibit occasional bursty behavior, can benefit from short performance boosts (Zheng and Wang, 2015).

Techniques like computational sprinting, that push the limits of what the hardware is designed to do, are conservative with respect to the safety of computations. This is mainly because exceeding hardware specifications leads to system instability. For instance, overclocking or activating a large number of cores for long periods leads to overheating, and exceeding the Thermal Design Power (TDP) of the circuit may cause faults (Nightingale et al., 2011), reduce the lifespan, or even physically damage the chip (Intel, 2017). Hence, after each sprint the CPU ought to switch to a cool-off mode.

Notwithstanding, even outside stable configurations, overclocking has been explored with interesting results. For instance, DSP-accelerators implemented with FPGAs can save up to 39% on hardware resources by overclocking for the same output quality (Duarte and Bouganis, 2015). It was also observed that as the frequency increases the errors in computations gradually appear in the output, up to the point where the program stops producing meaningful results.

In this paper, we envision bringing together these two vectors, by leveraging the redundancy that is already present in a large fraction of data center systems to safely push the limits of computational sprinting in data center environments. In other words, our goal is to aggressively explore overclocking settings, while taking advantage of the existing redundancy introduced by fault tolerance protocols to systematically mask faults that surface, in order to extend the benefits of sprinting (energy efficiency/performance). Furthermore, we intend to explore the synergies and subtle interactions between the two vectors. For instance, the sprints can be coordinated in a way that a subset of the replicas uses a more aggressive but unsafe sprinting to decrease the overall latency, but there is also a sufficient number of non-sprinted replicas that check the results and ensure both availability and correctness in the case of faults.

The remainder of the paper provides an overview of the key techniques that we can leverage as building blocks.

2. Computational sprinting

Advances in CMOS technology have enabled the design of modern multi-core processors, packing an increasing density of transistors at each new generation. However, more transistors implies an increase in power density, at a rate which exceeds the ability to dissipate the heat generated (Raghavan et al., 2013). As a result, continuously operating all the processing units at full power can permanently damage the chip due to overheating. Consequently, some of the cores must remain off most of the time, a limitation known as dark silicon (Esmaeilzadeh et al., 2011).

While it is not possible to activate all the processor cores at once in a sustainable way, there exist proposals for optimizing performance within safe temperature and power limits. These solutions (outlined next), leverage the fact that after activating a sprint, the temperature of the components does not rise instantaneously. Instead, it can take a few seconds for the heat to propagate through the chip package, allowing certain workloads to finish before it overheats.

Parallel sprinting. This approach consists of activating various dark silicon cores for up to a time limit (e.g., 1 sec.) before deactivating cores to cool off. Parallel sprinting can be optimized according to two policies: for maximum responsiveness, it activates all cores at maximum frequency and voltage; for optimal energy efficiency, it activates all cores at minimum frequency and voltage (Raghavan et al., 2012). Parallel sprinting is particularly interesting for mobile devices since a large part of mobile processors are comprised of accelerators that are inactive most of the time. In addition, mobile applications normally have interactive workloads that are characterized by short bursts and long idle times waiting for user input (Raghavan et al., 2013).

Note that there is an interesting research question, which we intend to explore, of whether data center workloads also have such characteristics. However, even if they do not, we can still attempt to split replicas in an alternating fashion between a group of replicas that are sprinting and another group that are cooling off.

Frequency Sprinting. A concrete example of frequency sprinting is the dynamic overclocking technology presented in commercial products such as Intel processors with Turbo Boost 2.0 (Intel, 2011). In a nutshell, Turbo Boost increases the frequencies and voltages of processor cores above a normal safe operation threshold for short periods. The sprint frequency target is defined automatically according to the available resources, allowing the processor to improve performance of both single and multi-threaded workloads. The algorithm that controls when sprinting is activated, takes into account the current frequencies of processor cores, the temperature of the package, and its power consumption (Intel, 2017).

Data center sprinting. Data centers can also experience bursty workloads, e.g., due to shared resources, maintenance activities, garbage collection events, or spikes in service popularity (Dean and Barroso, 2013; Gmach et al., 2007). Therefore, they might be a good match for sprinting. Moreover, the dark silicon phenomenon is likely to be prevalent in data centers, as supported by predictions that stated that by 2024 more than 50% of the chip must be powered off (Esmaeilzadeh et al., 2012). A current approach called Dynamo (Wu et al., 2016) employs power capping to enable frequency sprinting while ensuring that the energy drawn remains within the power budget limits. Alternatively, (Zheng and Wang, 2015) employs coordinated parallel sprinting, using the existing data center backup power supply (e.g. batteries), to provide the extra power for both sprint and cooling.

Overclocking. Pushing hardware to work beyond the prescribed frequency has been studied by other research communities (Shi et al., 2014). One approach is increasing the frequency, without sufficiently increasing the voltage, which can lead to faults, because it may violate the propagation delays of circuit critical paths. An interesting characteristic of these fault patterns is that errors gradually appear in the output as frequency increases. This gradual slope in the fault behavior happens because circuit designers are conservative in the estimates of path delays and keep a guard margin between the estimated clock frequency and the reported maximum clock frequency (Duarte and Bouganis, 2012), opening the opportunity to explore these limits.

3. Fault tolerance

Data center systems often rely on distributed replication protocols for operating correctly in the presence of faults. These protocols are designed under certain assumptions about the environment, such as fault behavior (e.g, crash, fail-stop, or Byzantine) and timing (e.g., the synchronous vs. the asynchronous model). Making wrong assumptions about the environment can either put the safety properties of the system at stake or impose an unnecessary cost in terms of replication and consequently energy and infrastructure.

A pragmatic choice for replicating services are asynchronous crash fault tolerant (CFT) protocols (Lamport, 1998; Ongaro and Ousterhout, 2014), because they cover the most common fault and timing behaviors. However, since aggressive overclocking can lead to data corruption or errors in a computation, CFT protocols may be too optimistic for sprinting. Byzantine fault tolerant (BFT) protocols (Castro and Liskov, 1999; Clement et al., 2009; Kapitza et al., 2012) are able to capture data corruption, but they are pessimistic regarding the behavior of faulty replicas. In particular, a BFT adversary may control a fraction of the replicas, allowing collusion, creating the worst case attack scenario. Such pessimism leads to a higher replication costs.

Visigoth fault tolerance (VFT) (Porto et al., 2015) allows for calibrating the fault tolerance of the system according to the deployment. While it can be configured to capture only crashes with the same replication requirements of CFT, VFT can also capture data corruption with a small additional cost. When compared to BFT, the replication requirements of VFT grows slower as the number of tolerated faults increases. This is because VFT assumes bounded collusion, i.e. the number of replicas that deviate from their expected behavior in the same way simultaneously. One of the reasons why this assumption may be realistic for overclocked systems is that the variations in the chip manufacturing process causes processors to have different stability configurations (Duarte and Bouganis, 2015). Additionally, we can enforce such diversity by carefully controlling overclocking at different replicas.

Other models have been proposed along the same lines as VFT, namely XFT (Liu et al., 2016), which has the same replication requirements as CFT while capturing Byzantine behavior, although not simultaneously with asynchrony. As future work, we can also explore whether this variant in the set of assumption is met in practice.

4. Conclusion

We presented our vision on the potential of combining computational sprinting and fault tolerance to enable higher savings on energy and improved performance for replicated systems. We intend to explore challenges of this approach, by designing and implementing a system that explores the opportunities identified here.

Acknowledgments

This research is funded by the European Research Council (ERC-2012-StG-307732) and by the FCT (UID/CEC/50021/2013).