Baseline: Operation-Based Evolution and Versioning of Data

TL;DR

Baseline introduces an operation-based approach to data versioning that enables fine-grained diffing, merging, and schema evolution management across diverse data structures, simplifying collaboration and change tracking.

Contribution

It presents a novel operational differencing technique for managing structural data changes, enabling version control without repositories and supporting flexible sharing and schema evolution.

Findings

Supports fine-grained diffing and merging despite schema changes

Enables simplified version control with no repositories or branching complexity

Rewrites queries to adapt to schema changes automatically

Abstract

Baseline is a platform for richly structured data supporting change in multiple dimensions: mutation over time, collaboration across space, and evolution through design changes. It is built upon Operational Differencing, a new technique for managing data in terms of high-level operations that include refactorings and schema changes. We use operational differencing to construct an operation-based form of version control on data structures used in programming languages and relational databases. This approach to data version control does fine-grained diffing and merging despite intervening structural transformations like schema changes. It offers users a simplified conceptual model of version control for ad hoc usage: There is no repo; Branching is just copying. The informaton maintained in a repo can be synthesized more precisely from the append-only histories of branches. Branches can be flexibly shared as is commonly done with document files, except with the added benefit of diffing and merging. We conjecture that queries can be operationalized into a sequence of schema and data operations. We develop that idea on a query language fragment containing selects and joins. Operationalized queries are represented as a future timeline that is speculatively executed as a branch off of the present state, returning a value from its hypothetical future. Operationalized queries get rewritten to accommodate schema change "for free" by the machinery of operational differencing. Altogether we develop solutions to four of the eight challenge problems of schema evolution identified in a recent paper.