FoundationDB Record Layer: A Multi-Tenant Structured Datastore

TL;DR

The paper introduces the FoundationDB Record Layer, an open source, multi-tenant, record-oriented data store built on FoundationDB, offering advanced schema, query, and indexing features for scalable cloud applications.

Contribution

It presents a highly extensible, stateless data store layer with unique features like nested types and cross-record indexes, optimized for multi-tenancy and cloud-scale use cases.

Findings

Used by Apple's CloudKit to host billions of databases.

Enables richer APIs with reduced maintenance.

Supports complex indexing and schema management.

Abstract

The FoundationDB Record Layer is an open source library that provides a record-oriented data store with semantics similar to a relational database implemented on top of FoundationDB, an ordered, transactional key-value store. The Record Layer provides a lightweight, highly extensible way to store structured data. It offers schema management and a rich set of query and indexing facilities, some of which are not usually found in traditional relational databases, such as nested record types, indexes on commit versions, and indexes that span multiple record types. The Record Layer is stateless and built for massive multi-tenancy, encapsulating and isolating all of a tenant's state, including indexes, into a separate logical database. We demonstrate how the Record Layer is used by CloudKit, Apple's cloud backend service, to provide powerful abstractions to applications serving hundreds of millions of users. CloudKit uses the Record Layer to host billions of independent databases, many with a common schema. Features provided by the Record Layer enable CloudKit to provide richer APIs and stronger semantics with reduced maintenance overhead and improved scalability.