Multi-modal Synthesis of Regular Expressions

TL;DR

This paper introduces Regel, a multi-modal approach combining natural language and examples to synthesize regular expressions more accurately and efficiently than existing methods, improving user success rates.

Contribution

The paper presents a novel hierarchical sketch-based synthesis technique that enhances regex generation by integrating natural language and examples, outperforming prior single-modality approaches.

Findings

Regel achieves 80% accuracy in regex synthesis.

Regel's PBE engine is an order of magnitude faster than AlphaRegex.

Users are twice as successful with Regel in a user study.

Abstract

In this paper, we propose a multi-modal synthesis technique for automatically constructing regular expressions (regexes) from a combination of examples and natural language. Using multiple modalities is useful in this context because natural language alone is often highly ambiguous, whereas examples in isolation are often not sufficient for conveying user intent. Our proposed technique first parses the English description into a so-called hierarchical sketch that guides our programming-by-example (PBE) engine. Since the hierarchical sketch captures crucial hints, the PBE engine can leverage this information to both prioritize the search as well as make useful deductions for pruning the search space. We have implemented the proposed technique in a tool called Regel and evaluate it on over three hundred regexes. Our evaluation shows that Regel achieves 80% accuracy whereas the NLP-only and PBE-only baselines achieve 43% and 26% respectively. We also compare our proposed PBE engine against an adaptation of AlphaRegex, a state-of-the-art regex synthesis tool, and show that our proposed PBE engine is an order of magnitude faster, even if we adapt the search algorithm of AlphaRegex to leverage the sketch. Finally, we conduct a user study involving 20 participants and show that users are twice as likely to successfully come up with the desired regex using Regel compared to without it.