MigCast in Monte Carlo: The Impact of Data Model Evolution in NoSQL Databases

TL;DR

This paper investigates how data model evolution affects migration costs and latency in NoSQL databases, using Monte Carlo simulations to analyze various migration scenarios and inform software release strategies.

Contribution

It introduces a probabilistic Monte Carlo approach to evaluate the impact of schema changes on migration costs and performance in NoSQL databases, aiding decision-making.

Findings

Migration costs vary significantly with data access patterns and schema change types.

Monte Carlo sampling effectively manages the complexity of migration scenario analysis.

The study provides insights into optimizing release strategies to balance migration costs and latency.

Abstract

During the development of NoSQL-backed software, the data model evolves naturally alongside the application code. Especially in agile development, new application releases are deployed frequently causing schema changes. Eventually, decisions have to be made regarding the migration of versioned legacy data which is persisted in the cloud-hosted production database. We solve this schema evolution problem and present the results of near-exhaustive calculations by means of which software project stakeholders can manage the operative costs for data model evolution and adapt their software release strategy accordingly in order to comply with service-level agreements regarding the competing metrics of migration costs and latency. We clarify conclusively how data model evolution in NoSQL databases impacts the metrics while taking all relevant characteristics of migration scenarios into account. As calculating all possible combinatorics in the search space of migration scenarios would by far exceed computational means, we used a probabilistic Monte Carlo method of repeated sampling, serving as a well-established means to bring the complexity of data model evolution under control. Our experiments show the qualitative and quantitative impact on the performance of migration strategies with respect to intensity and distribution of data entity accesses, the kinds of schema changes, and the characteristics of the underlying data model.