SwitchFS: Asynchronous Metadata Updates for Distributed Filesystems with In-Network Coordination

TL;DR

SwitchFS introduces asynchronous metadata updates in distributed filesystems, leveraging programmable switches to improve throughput and latency under skewed workloads by efficiently batching and applying delayed updates.

Contribution

This paper presents SwitchFS, a novel approach that uses in-network coordination with programmable switches to enable asynchronous metadata updates while maintaining POSIX semantics.

Findings

Achieves up to 13.34× higher throughput compared to state-of-the-art systems.

Reduces latency by up to 61.6% under skewed workloads.

Improves real-world workload throughput significantly, up to 21.1×.

Abstract

Distributed filesystem metadata updates are typically synchronous. This creates inherent challenges for access efficiency, load balancing, and directory contention, especially under dynamic and skewed workloads. This paper argues that synchronous updates are overly conservative. We propose SwitchFS with asynchronous metadata updates that allow operations to return early and defer directory updates until reads, both hiding latency and amortizing overhead. The key challenge lies in efficiently maintaining the synchronous POSIX semantics of metadata updates. To address this, SwitchFS is co-designed with a programmable switch, leveraging the limited on-switch resources to track directory states with negligible overhead. This allows SwitchFS to aggregate and apply delayed updates efficiently, using batching and consolidation before directory reads. Evaluation shows that SwitchFS achieves up to 13.34$\times$ and 3.85$\times$ higher throughput, and 61.6% and 57.3% lower latency than two state-of-the-art distributed filesystems, Emulated-InfiniFS and Emulated-CFS, respectively, under skewed workloads. For real-world workloads, SwitchFS improves end-to-end throughput by 21.1$\times$, 1.1$\times$, and 0.3$\times$ over CephFS, Emulated-InfiniFS, and Emulated-CFS, respectively.