The Open Cloud Testbed: A Wide Area Testbed for Cloud Computing Utilizing High Performance Network Services

TL;DR

The paper introduces the Open Cloud Testbed, a large-scale, wide-area cloud computing infrastructure with high-performance networking, designed for benchmarking, interoperability testing, and developing new cloud services.

Contribution

It presents the design, implementation, and utilities of the OCT, a unique wide-area cloud testbed with dedicated high-speed networks and tools for cloud system development and testing.

Findings

Successfully deployed 120 nodes across four data centers

Developed benchmarks and interoperability studies for cloud systems

Established utilities for node/network provisioning and monitoring

Abstract

Recently, a number of cloud platforms and services have been developed for data intensive computing, including Hadoop, Sector, CloudStore (formerly KFS), HBase, and Thrift. In order to benchmark the performance of these systems, to investigate their interoperability, and to experiment with new services based on flexible compute node and network provisioning capabilities, we have designed and implemented a large scale testbed called the Open Cloud Testbed (OCT). Currently the OCT has 120 nodes in four data centers: Baltimore, Chicago (two locations), and San Diego. In contrast to other cloud testbeds, which are in small geographic areas and which are based on commodity Internet services, the OCT is a wide area testbed and the four data centers are connected with a high performance 10Gb/s network, based on a foundation of dedicated lightpaths. This testbed can address the requirements of extremely large data streams that challenge other types of distributed infrastructure. We have also developed several utilities to support the development of cloud computing systems and services, including novel node and network provisioning services, a monitoring system, and a RPC system. In this paper, we describe the OCT architecture and monitoring system. We also describe some benchmarks that we developed and some interoperability studies we performed using these benchmarks.