An Exploratory Study of How Specialists Deal with Testing in Data Stream Processing Applications

TL;DR

This paper explores current testing practices for data stream processing applications through interviews and surveys, providing a comprehensive overview of methodologies, techniques, and tools used by practitioners.

Contribution

It offers the first consolidated, practice-based overview of testing approaches in data stream processing, filling a significant literature gap.

Findings

Practitioners use diverse methodologies and tools for testing data streams.

There is no single standard approach; practices vary by context.

The study provides a compendium of testing alternatives for streaming applications.

Abstract

[Background] Nowadays, there is a massive growth of data volume and speed in many types of systems. It introduces new needs for infrastructure and applications that have to handle streams of data with low latency and high throughput. Testing applications that process such data streams has become a significant challenge for engineers. Companies are adopting different approaches to dealing with this issue. Some have developed their own solutions for testing, while others have adopted a combination of existing testing techniques. There is no consensus about how or in which contexts such solutions can be implemented. [Aims] To the best of our knowledge, there is no consolidated literature on that topic. The present paper is an attempt to fill this gap by conducting an exploratory study with practitioners. [Method] We used qualitative methods in this research, in particular interviews and survey. We interviewed 12 professionals who work in projects related to data streams, and also administered a questionnaire with other 105 professionals. The interviews went through a transcription and coding process, and the questionnaires were analysed to reinforce findings. [Results] This study presents current practices around software testing in data stream processing applications. These practices involve methodologies, techniques, and tools. [Conclusions] Our main contribution is a compendium of alternatives for many of the challenges that arise when testing streaming applications from a state-of-the-practice perspective.