Report on the XBase Project

TL;DR

This paper explores the fundamental concepts of flexible, secure, and structure-neutral data storage systems, emphasizing separation of mechanism and policy, with key advances in minimal building blocks and append-only storage insights.

Contribution

It introduces a set of minimal, composable storage system components and provides insights into append-only storage challenges for flexible data management.

Findings

Identified minimal storage system building blocks.

Developed a conceptual framework for flexible storage architectures.

Analyzed issues related to append-only storage components.

Abstract

This project addressed the conceptual fundamentals of data storage, investigating techniques for provision of highly generic storage facilities that can be tailored to produce various individually customised storage infrastructures, compliant to the needs of particular applications. This requires the separation of mechanism and policy wherever possible. Aspirations include: actors, whether users or individual processes, should be able to bind to, update and manipulate data and programs transparently with respect to their respective locations; programs should be expressed independently of the storage and network technology involved in their execution; storage facilities should be structure-neutral so that actors can impose multiple interpretations over information, simultaneously and safely; information should not be discarded so that arbitrary historical views are supported; raw stored information should be open to all; where security restrictions on its use are required this should be achieved using cryptographic techniques. The key advances of the research were: 1) the identification of a candidate set of minimal storage system building blocks, which are sufficiently simple to avoid encapsulating policy where it cannot be customised by applications, and composable to build highly flexible storage architectures 2) insight into the nature of append-only storage components, and the issues arising from their application to common storage use-cases.