Maxwell: a hardware and software highly integrated compute-storage system

TL;DR

Maxwell is a highly integrated compute-storage system designed to address hot-spot account update bottlenecks, achieving significant performance improvements through hardware-software co-optimization without relying on specialized hardware.

Contribution

The paper introduces Maxwell, a novel distributed compute-storage system with integrated hardware and software optimizations that significantly enhances performance for hot-spot account updates.

Findings

Achieves three orders of magnitude performance improvement over existing solutions.

Demonstrates remarkable performance gains in Ant Group's related businesses.

Does not depend on specialized hardware like GPUs or FPGAs.

Abstract

The compute-storage framework is responsible for data storage and processing, and acts as the digital chassis of all upper-level businesses. The performance of the framework affects the business's processing throughput, latency, jitter, and etc., and also determines the theoretical performance upper bound that the business can achieve. In financial applications, the compute-storage framework must have high reliability and high throughput, but with low latency as well as low jitter characteristics. For some scenarios such as hot-spot account update, the performance of the compute-storage framework even surfaces to become a server performance bottleneck of the whole business system. In this paper, we study the hot-spot account issue faced by Alipay and present our exciting solution to this problem by developing a new compute-storage system, called Maxwell. Maxwell is a distributed compute-storage system with integrated hardware and software optimizations. Maxwell does not rely on any specific hardware (e.g. GPUs or FPGAs). Instead, it takes deep advantage of computer components' characteristics, such as disk, network, operating system and CPU, and aims to emit the ultimate performance of both hardware and software. In comparison with the existing hot-spot account updating solutions deployed online, Maxwell achieves three orders of magnitude performance improvement for end-to-end evaluation. Meanwhile, Maxwell also demonstrates remarkable performance gains in other related businesses of Ant Group.