ICST Tool Competition 2025 -- Self-Driving Car Testing Track

TL;DR

This paper introduces the first ICST tool competition focused on testing self-driving cars, providing a platform and case studies for developing and evaluating test selection tools in simulation-based testing.

Contribution

It establishes a new competition framework for SDC testing tools, including evaluation methods and baseline comparisons, fostering progress in this emerging domain.

Findings

Five tools evaluated using regression metrics

Tools compared with baseline approaches

Insights into effectiveness of different test selection strategies

Abstract

This is the first edition of the tool competition on testing self-driving cars (SDCs) at the International Conference on Software Testing, Verification and Validation (ICST). The aim is to provide a platform for software testers to submit their tools addressing the test selection problem for simulation-based testing of SDCs, which is considered an emerging and vital domain. The competition provides an advanced software platform and representative case studies to ease participants' entry into SDC regression testing, enabling them to develop their initial test generation tools for SDCS. In this first edition, the competition includes five tools from different authors. All tools were evaluated using (regression) metrics for test selection as well as compared with a baseline approache. This paper provides an overview of the competition, detailing its context, framework, participating tools, evaluation methodology, and key findings.