DataRaceBench V1.4.1 and DataRaceBench-ML V0.1: Benchmark Suites for Data Race Detection

TL;DR

This paper updates the DataRaceBench suite with new data race data and introduces DataRaceBench-ML, a dataset tailored for machine learning applications in data race detection, including detailed labels and prompt-response pairs for LLM fine-tuning.

Contribution

It presents an updated benchmark suite and a new ML-oriented dataset derived from it, facilitating machine learning research in data race detection.

Findings

Enhanced DataRaceBench with new data race data

Introduction of DataRaceBench-ML dataset for ML applications

Inclusion of prompt-response pairs for LLM fine-tuning

Abstract

Data races pose a significant threat in multi-threaded parallel applications due to their negative impact on program correctness. DataRaceBench, an open-source benchmark suite, is specifically crafted to assess these data race detection tools in a systematic and measurable manner. Machine learning techniques have recently demonstrated considerable potential in high-performance computing (HPC) program analysis and optimization. However, these techniques require specialized data formats for training and refinement. This paper presents the latest update to DataRaceBench, incorporating new data race contributions from Wu et al. \cite{wu2023model}, and introduces a derived dataset named DataRaceBench-ML (DRB-ML) \cite{drbml}. DRB-ML aligns with the emerging trend of machine learning and large language models. Originating from DataRaceBench, this dataset includes detailed labels that denote the presence of a data race and provides comprehensive details of associated variables, such as variable names, line numbers, and the operation (read/write). Unique to DRB-ML, we have also integrated a series of tailored prompt-response pairs specifically designed for LLM fine-tuning.