An architecture for enabling A/B experiments in automotive embedded software

TL;DR

This paper presents an architecture designed to facilitate A/B testing in automotive embedded software, addressing industry-specific challenges and demonstrating practical application and relevance for continuous experimentation.

Contribution

It introduces a novel architecture tailored for automotive embedded software to enable systematic A/B experimentation, filling a gap in current industry practices.

Findings

Architecture effectively supports large-scale online experiments

Proven applicability through real-world case study

Enhances continuous improvement in automotive software development

Abstract

A/B experimentation is a known technique for data-driven product development and has demonstrated its value in web-facing businesses. With the digitalisation of the automotive industry, the focus in the industry is shifting towards software. For automotive embedded software to continuously improve, A/B experimentation is considered an important technique. However, the adoption of such a technique is not without challenge. In this paper, we present an architecture to enable A/B testing in automotive embedded software. The design addresses challenges that are unique to the automotive industry in a systematic fashion. Going from hypothesis to practice, our architecture was also applied in practice for running online experiments on a considerable scale. Furthermore, a case study approach was used to compare our proposal with state-of-practice in the automotive industry. We found our architecture design to be relevant and applicable in the efforts of adopting continuous A/B experiments in automotive embedded software.