Duplicate-sensitivity Guided Transformation Synthesis for DBMS Correctness Bug Detection

TL;DR

This paper presents an automated method for generating query transformations that detect correctness bugs in DBMSs by ensuring duplicate sensitivity, leading to the discovery of new bugs in popular systems.

Contribution

It introduces a duplicate-sensitivity guided transformation synthesis approach that automates the creation of equivalent query pairs for testing DBMS correctness.

Findings

Detected 30 new bugs in MySQL, TiDB, and CynosDB

Automated transformation synthesis reduces manual effort

Effective in uncovering correctness bugs in production DBMSs

Abstract

Database Management System (DBMS) plays a core role in modern software from mobile apps to online banking. It is critical that DBMS should provide correct data to all applications. When the DBMS returns incorrect data, a correctness bug is triggered. Current production-level DBMSs still suffer from insufficient testing due to the limited hand-written test cases. Recently several works proposed to automatically generate many test cases with query transformation, a process of generating an equivalent query pair and testing a DBMS by checking whether the system returns the same result set for both queries. However, all of them still heavily rely on manual work to provide a transformation which largely confines their exploration of the valid input query space. This paper introduces duplicate-sensitivity guided transformation synthesis which automatically finds new transformations by first synthesizing many candidates then filtering the nonequivalent ones. Our automated synthesis is achieved by mutating a query while keeping its duplicate sensitivity, which is a necessary condition for query equivalence. After candidate synthesis, we keep the mutant query which is equivalent to the given one by using a query equivalent checker. Furthermore, we have implemented our idea in a tool Eqsql and used it to test the production-level DBMSs. In two months, we detected in total 30 newly confirmed and unique bugs in MySQL, TiDB and CynosDB.