Towards Marrying Files to Objects

TL;DR

This paper explores the feasibility of integrating object-based storage with traditional file systems to create a unified platform that preserves existing semantics and interfaces, addressing storage sprawl.

Contribution

It presents an analysis of design tradeoffs for merging object stores and file systems and demonstrates a POSIX-complete implementation highlighting key considerations.

Findings

Unified storage can potentially reduce data management complexity.

Design tradeoffs impact performance and compatibility.

Experimental implementation reveals critical design considerations.

Abstract

To deal with the constant growth of unstructured data, vendors have deployed scalable, resilient, and cost effective object-based storage systems built on RESTful web services. However, many applications rely on richer file-system APIs and semantics, and cannot benefit from object stores. This leads to storage sprawl, as object stores are deployed alongside file systems and data is accessed and managed across both systems in an ad-hoc fashion. We believe there is a critical need for a transparent merger of objects and files, consolidating data into a single platform. Such a merger would extend the capabilities of both object and file stores while preserving existing semantics and interfaces. In this position paper, we examine the viability of unifying object stores and file systems, and the various design tradeoffs that exist. Then, using our own implementation of an object-based, POSIX-complete file system, we experimentally demonstrate several critical design considerations.