A Benchmarking Framework for Interactive 3D Applications in the Cloud

TL;DR

This paper introduces Pictor, a benchmarking framework employing AI to simulate human interactions and analyze performance in cloud-based 3D graphics applications, aiding in identifying bottlenecks and optimizing system performance.

Contribution

The paper presents Pictor, a novel AI-driven infrastructure for benchmarking and performance analysis of interactive 3D applications in the cloud, including a new benchmark suite and optimization techniques.

Findings

Pictor effectively mimics human interactions in 3D applications.

Performance bottlenecks in cloud 3D graphics systems can be identified and addressed.

Optimizations improved frame rate by 57.7% on average.

Abstract

With the growing popularity of cloud gaming and cloud virtual reality (VR), interactive 3D applications have become a major type of workloads for the cloud. However, despite their growing importance, there is limited public research on how to design cloud systems to efficiently support these applications, due to the lack of an open and reliable research infrastructure, including benchmarks and performance analysis tools. The challenges of generating human-like inputs under various system/application randomness and dissecting the performance of complex graphics systems make it very difficult to design such an infrastructure. In this paper, we present the design of a novel cloud graphics rendering research infrastructure, Pictor. Pictor employs AI to mimic human interactions with complex 3D applications. It can also provide in-depth performance measurements for the complex software and hardware stack used for cloud 3D graphics rendering. With Pictor, we designed a benchmark suite with six interactive 3D applications. Performance analyses were conducted with these benchmarks to characterize 3D applications in the cloud and reveal new performance bottlenecks. To demonstrate the effectiveness of Pictor, we also implemented two optimizations to address two performance bottlenecks discovered in a state-of-the-art cloud 3D-graphics rendering system, which improved the frame rate by 57.7% on average.