NARRepair: Non-Autoregressive Code Generation Model for Automatic Program Repair

TL;DR

NARRepair introduces a non-autoregressive model for automatic program repair, significantly reducing inference time while maintaining high accuracy by leveraging repair actions, AST dependency info, and two-stage decoding.

Contribution

This paper presents the first NAR-based APR model, NARRepair, with novel techniques to improve speed and accuracy over traditional AR methods.

Findings

Significantly faster inference compared to AR models.

Maintains high repair accuracy on multiple datasets.

Effectively utilizes AST dependency and repair actions.

Abstract

With the advancement of deep learning techniques, the performance of Automatic Program Repair(APR) techniques has reached a new level. Previous deep learning-based APR techniques essentially modified program sentences in the Autoregressive(AR) manner, which predicts future values based on past values. Due to the manner of word-by-word generation, the AR-based APR technique has a huge time delay. This negative consequence overshadows the widespread adoption of APR techniques in real-life software development. To address the issue, we aim to apply the Non-Autoregressive(NAR) method to the APR task, which can output target code in a parallel manner to avoid huge inference delays. To effectively adapt the NAR manner for the APR task, we in this paper propose NARRepair, the first customized NAR code generation model for the APR task. The NARRepair features three major novelties, including 1) using repair actions to alleviate the over-correction issue, 2) extracting dependency information from AST to alleviate the issue of lacking inter-word dependency information, 3) employing two-stage decoding to alleviate the issue of lacking contextual information. We evaluated NARRepair on three widely used datasets in the APR community, and the results show that our technique can significantly improve the inference speed while maintaining high repair accuracy.