We'll Fix It in Post: What Do Bug Fixes in Video Game Update Notes Tell Us?

TL;DR

This study analyzes bug fix notes from 30 popular Steam games to understand bug types, recurrence, severity, and developer perceptions, aiming to improve game testing and bug prevention strategies.

Contribution

It introduces a bug taxonomy for games, analyzes a large dataset of bug fixes, and surveys developers to link bug types with development challenges and recurrence.

Findings

Information bugs are most frequent in updates.

Crash bugs recur most often and are deemed more severe.

Testing challenges and code quality issues are linked to bug persistence.

Abstract

Bugs that persist into releases of video games can have negative impacts on both developers and users, but particular aspects of testing in game development can lead to difficulties in effectively catching these missed bugs. It has become common practice for developers to apply updates to games in order to fix missed bugs. These updates are often accompanied by notes that describe the changes to the game included in the update. However, some bugs reappear even after an update attempts to fix them. In this paper, we develop a taxonomy for bug types in games that is based on prior work. We examine 12,122 bug fixes from 723 updates for 30 popular games on the Steam platform. We label the bug fixes included in these updates to identify the frequency of these different bug types, the rate at which bug types recur over multiple updates, and which bug types are treated as more severe. Additionally, we survey game developers regarding their experience with different bug types and what aspects of game development they most strongly associate with bug appearance. We find that Information bugs appear the most frequently in updates, while Crash bugs recur the most frequently and are often treated as more severe than other bug types. Finally, we find that challenges in testing, code quality, and bug reproduction have a close association with bug persistence. These findings should help developers identify which aspects of game development could benefit from greater attention in order to prevent bugs. Researchers can use our results in devising tools and methods to better identify and address certain bug types.