Understanding and Reusing Test Suites Across Database Systems
Suyang Zhong, Manuel Rigger

TL;DR
This paper investigates the challenges and potential of reusing extensive test suites across different database management systems, highlighting variability in formats, dependencies, and non-standardized tests, while demonstrating some benefits of reuse through empirical analysis.
Contribution
It introduces a unified test suite, SQuaLity, combining test cases from SQLite, PostgreSQL, and DuckDB, and provides an empirical study on the feasibility and challenges of test reuse across DBMSs.
Findings
Test formats and runner commands vary widely across systems.
Some test suites are mostly standard-compliant, others are not.
Test reuse can reveal crashes, hangs, and compatibility issues.
Abstract
Database Management System (DBMS) developers have implemented extensive test suites to test their DBMSs. For example, the SQLite test suites contain over 92 million lines of code. Despite these extensive efforts, test suites are not systematically reused across DBMSs, leading to wasted effort. Integration is challenging, as test suites use various test case formats and rely on unstandardized test runner features. We present a unified test suite, SQuaLity, in which we integrated test cases from three widely-used DBMSs, SQLite, PostgreSQL, and DuckDB. In addition, we present an empirical study to determine the potential of reusing these systems' test suites. Our results indicate that reusing test suites is challenging: First, test formats and test runner commands vary widely; for example, SQLite has 4 test runner commands, while MySQL has 112 commands with additional features, to, for…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
