CFS: A Distributed File System for Large Scale Container Platforms

TL;DR

CFS is a new distributed file system optimized for large-scale container platforms, offering improved performance for file access, replication, and metadata management compared to existing solutions like Ceph.

Contribution

CFS introduces a novel metadata placement strategy and tailored replication protocols, enhancing performance and scalability for container platform storage.

Findings

CFS achieves approximately 3x performance boost over Ceph in metadata operations.

CFS provides better random read/write performance under high concurrency.

CFS supports both sequential and random file access with optimized storage.

Abstract

We propose CFS, a distributed file system for large scale container platforms. CFS supports both sequential and random file accesses with optimized storage for both large files and small files, and adopts different replication protocols for different write scenarios to improve the replication performance. It employs a metadata subsystem to store and distribute the file metadata across different storage nodes based on the memory usage. This metadata placement strategy avoids the need of data rebalancing during capacity expansion. CFS also provides POSIX-compliant APIs with relaxed semantics and metadata atomicity to improve the system performance. We performed a comprehensive comparison with Ceph, a widely-used distributed file system on container platforms. Our experimental results show that, in testing 7 commonly used metadata operations, CFS gives around 3 times performance boost on average. In addition, CFS exhibits better random-read/write performance in highly concurrent environments with multiple clients and processes.