TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest

TL;DR

This paper introduces TestMigrationsInPy, a comprehensive dataset of 923 real-world Python test migrations from unittest to pytest, aimed at facilitating automated migration solutions and research validation.

Contribution

The paper presents a new dataset of real-world test migrations from unittest to pytest, including detailed migration types, supporting future research and tool development.

Findings

Contains 923 real-world migration cases

Includes detailed migration type annotations

Enables validation of migration tools

Abstract

Unittest and pytest are the most popular testing frameworks in Python. Overall, pytest provides some advantages, including simpler assertion, reuse of fixtures, and interoperability. Due to such benefits, multiple projects in the Python ecosystem have migrated from unittest to pytest. To facilitate the migration, pytest can also run unittest tests, thus, the migration can happen gradually over time. However, the migration can be time-consuming and take a long time to conclude. In this context, projects would benefit from automated solutions to support the migration process. In this paper, we propose TestMigrationsInPy, a dataset of test migrations from unittest to pytest. TestMigrationsInPy contains 923 real-world migrations performed by developers. Future research proposing novel solutions to migrate frameworks in Python can rely on TestMigrationsInPy as a ground truth. Moreover, as TestMigrationsInPy includes information about the migration type (e.g., changes in assertions or fixtures), our dataset enables novel solutions to be verified effectively, for instance, from simpler assertion migrations to more complex fixture migrations. TestMigrationsInPy is publicly available at: https://github.com/altinoalvesjunior/TestMigrationsInPy.