InfeRE: Step-by-Step Regex Generation via Chain of Inference

TL;DR

InfeRE introduces a step-by-step inference approach for regex generation from natural language, improving accuracy and interpretability over previous autoregressive methods by decomposing the process and using ensemble decoding.

Contribution

The paper proposes a novel chain-of-inference paradigm for regex generation, enhancing robustness and performance compared to existing single-pass models.

Findings

Achieves 16.3% and 14.7% improvements in DFA@5 accuracy on two datasets.

Outperforms state-of-the-art approaches and TRANX by significant margins.

Demonstrates the effectiveness of step-by-step inference and ensemble decoding in regex generation.

Abstract

Automatically generating regular expressions (abbrev. regexes) from natural language description (NL2RE) has been an emerging research area. Prior studies treat regex as a linear sequence of tokens and generate the final expressions autoregressively in a single pass. They did not take into account the step-by-step internal text-matching processes behind the final results. This significantly hinders the efficacy and interpretability of regex generation by neural language models. In this paper, we propose a new paradigm called InfeRE, which decomposes the generation of regexes into chains of step-by-step inference. To enhance the robustness, we introduce a self-consistency decoding mechanism that ensembles multiple outputs sampled from different models. We evaluate InfeRE on two publicly available datasets, NL-RX-Turk and KB13, and compare the results with state-of-the-art approaches and the popular tree-based generation approach TRANX. Experimental results show that InfeRE substantially outperforms previous baselines, yielding 16.3% and 14.7% improvement in DFA@5 accuracy on two datasets, respectively. Particularly, InfeRE outperforms the popular tree-based generation approach by 18.1% and 11.3% on both datasets, respectively, in terms of DFA@5 accuracy.