# Identifying Patch Correctness in Test-Based Program Repair

## TL;DR

This paper introduces a heuristic method that uses behavior similarity of test executions to identify correct patches in test-based program repair, significantly reducing incorrect patches.

## Contribution

The paper presents a novel approach leveraging test behavior similarity and input generation to improve correctness verification in automatic program repair.

## Key findings

- Prevented 56.3% of incorrect patches
- Did not block any correct patches
- Effective across multiple repair systems

## Abstract

Test-based automatic program repair has attracted a lot of attention in recent years. However, the test suites in practice are often too weak to guarantee correctness and existing approaches often generate a large number of incorrect patches. To reduce the number of incorrect patches generated, we propose a novel approach that heuristically determines the correctness of the generated patches. The core idea is to exploit the behavior similarity of test case executions. The passing tests on original and patched programs are likely to behave similarly while the failing tests on original and patched programs are likely to behave differently. Also, if two tests exhibit similar runtime behavior, the two tests are likely to have the same test results. Based on these observations, we generate new test inputs to enhance the test suites and use their behavior similarity to determine patch correctness. Our approach is evaluated on a dataset consisting of 139 patches generated from existing program repair systems including jGenProg, Nopol, jKali, ACS and HDRepair. Our approach successfully prevented 56.3\% of the incorrect patches to be generated, without blocking any correct patches.