Extending Hierarchical Delta Debugging with Hoisting

TL;DR

This paper introduces an extension to Hierarchical Delta Debugging by incorporating hoisting, a technique that replaces subtrees with compatible ones to achieve more significant test case reductions, improving efficiency and output size.

Contribution

The paper proposes a novel extension to HDD called hoisting, which replaces subtrees with compatible ones, leading to up to 80% smaller test cases and outperforming existing algorithms.

Findings

Hoisting can reduce test case size by up to 80%.

Extended HDD produces smaller test cases than state-of-the-art algorithms.

Hoisting improves the efficiency of test case minimization.

Abstract

Minimizing failing test cases is an important pre-processing step on the path of debugging. If much of a test case that triggered a bug does not contribute to the actual failure, then the time required to fix the bug can increase considerably. However, test case reduction itself can be a time consuming task, especially if done manually. Therefore, automated minimization techniques have been proposed, the minimizing Delta Debugging (DDMIN) and the Hierarchical Delta Debugging (HDD) algorithms being the most well known. DDMIN does not need any information about the structure of the test case, thus it works for any kind of input. If the structure is known, however, it can be utilized to create smaller test cases faster. This is exemplified by HDD, which works on tree-structured inputs, pruning subtrees at each level of the tree with the help of DDMIN. In this paper, we propose to extend HDD with a reduction method that does not prune subtrees, but replaces them with compatible subtrees further down the hierarchy, called hoisting. We have evaluated various combinations of pruning and hoisting on multiple test suites and found that hoisting can help to further reduce the size of test cases by as much as 80% compared to the baseline HDD. We have also compared our results to other state-of-the-art test case reduction algorithms and found that HDD extended with hoisting can produce smaller output in most of the cases.