Black-Box Bug-Amplification for Multithreaded Software

TL;DR

This paper introduces a black-box approach to amplify elusive concurrency bugs in multithreaded software by using predictive models to guide repeated testing, significantly increasing bug detection rates without modifying system internals.

Contribution

It presents a novel formulation of bug amplification as a rare-event regression problem and evaluates model-guided search techniques for effective bug detection.

Findings

Ensemble regression models improve bug occurrence rates by up to an order of magnitude.

Model-guided search outperforms random sampling in bug amplification.

The approach is practical and non-invasive, suitable for real-world testing.

Abstract

Bugs, especially those in concurrent systems, are often hard to reproduce because they manifest only under rare conditions. Testers frequently encounter failures that occur only under specific inputs, even when occurring with low probability. We propose an approach to systematically amplify the occurrence of such elusive bugs. We treat the system under test as a black-box and use repeated trial executions to train a predictive model that estimates the probability of a given input configuration triggering a bug. We evaluate this approach on a dataset of 17 representative concurrency bugs spanning diverse categories. Several model-based search techniques are compared against a brute-force random sampling baseline. Our results show that an ensemble of regression models can significantly increase bug occurrence rates across nearly all scenarios, often achieving an order-of-magnitude improvement over random sampling. The contributions of this work include: (i) a novel formulation of bug-amplification as a rare-event regression problem; (ii) an empirical evaluation of multiple techniques for amplifying bug occurrence, demonstrating the effectiveness of model-guided search; and (iii) a practical, non-invasive testing framework that helps practitioners expose hidden concurrency faults without altering the internal system architecture.