A Study of Real-World Data Races in Golang

TL;DR

This paper investigates real-world data races in Go by analyzing Uber's extensive codebase, revealing the prevalence, patterns, and challenges of detecting and fixing data races in industry-scale systems.

Contribution

It provides the first large-scale empirical study of data races in Go, highlighting language-specific issues and practical engineering trade-offs in using dynamic race detectors.

Findings

Over 2000 data races found in Uber's Go codebase

Fixed over 1000 data races across 790 code patches

Identified complex patterns and language-specific challenges in data race detection

Abstract

The concurrent programming literature is rich with tools and techniques for data race detection. Less, however, has been known about real-world, industry-scale deployment, experience, and insights about data races. Golang (Go for short) is a modern programming language that makes concurrency a first-class citizen. Go offers both message passing and shared memory for communicating among concurrent threads. Go is gaining popularity in modern microservice-based systems. Data races in Go stand in the face of its emerging popularity. In this paper, using our industrial codebase as an example, we demonstrate that Go developers embrace concurrency and show how the abundance of concurrency alongside language idioms and nuances make Go programs highly susceptible to data races. Google's Go distribution ships with a built-in dynamic data race detector based on ThreadSanitizer. However, dynamic race detectors pose scalability and flakiness challenges; we discuss various software engineering trade-offs to make this detector work effectively at scale. We have deployed this detector in Uber's 46 million lines of Go codebase hosting 2100 distinct microservices, found over 2000 data races, and fixed over 1000 data races, spanning 790 distinct code patches submitted by 210 unique developers over a six-month period. Based on a detailed investigation of these data race patterns in Go, we make seven high-level observations relating to the complex interplay between the Go language paradigm and data races.