8 Years of Optimizing Apache Otava: How disconnected open source developers took an algorithm from n3 to constant time

TL;DR

Over 8 years, disconnected open source developers optimized Apache Otava from an O(n^3) to an O(1) algorithm, enabling real-time change point detection and significantly enhancing performance.

Contribution

The paper documents the historical development and introduces the seventh generation of performance optimizations for Apache Otava, achieving a transition from cubic to constant time complexity.

Findings

18,000 to 300,000 times speedup over original implementation

Achieved constant time complexity (O(1)) for change point detection

Enabled real-time computation and user experience improvements

Abstract

As the project now known as Apache Otava (incubating) makes it first release, we look back over the past 8 years that the codebase was developed by a rather uncoordinated, loosely connected group of performance engineers at MongoDB, Datastax, Confluent, Nyrkio and others. Ever since the first publication (Daly 2020), developers of the code base now known as Apache Otava (incubating), have continuosly improved its performance. Even when a contributor's primary motivation was to add functionality, it seems like they couldn't help themselves but to also make some performance optimizations while at it. When developing the Nyrkio web service to provide change detection for performance testing, we have observed that Otava had become fast enough that it was almost feasible to compute change points synchronously, as the user is browsing test results in a web browser. Inspired by this, we have developed and contributed a new optimization for the common case where new data points are appended to the end of the series. This is now the 7th generation of performance optimizations in Otava. These improvements have been done over the past 8 years of development, by disconnected individuals at different employees. Taken together, the historical optimizations and those published in this paper, represent a 18 000 to 300 000 speedup over the original by the book implementation of (Matteson and James 2014). In the language of computational complexity, an evolution from O (n3) to O (1) (constant time). The ability to compute and recompute change points in real-time unlocks new opportunities in the user experience.