Syntactic Completions with Material Obligations

TL;DR

This paper introduces talltylr, a novel parser and editor generator that handles syntax errors by completing malformed code with syntactic obligations, improving error recovery and developer assistance.

Contribution

It presents a new tile-based parsing theory and implementation that generalizes syntax error recovery through obligations, with formal guarantees and visual materialization for better usability.

Findings

Obligations help scaffold internal program representations.

Materializing obligations enhances user understanding and editing experience.

Formal guarantees ensure successful parsing over all inputs.

Abstract

Code editors provide essential services that help developers understand, navigate, and modify programs. However, these services often fail in the presence of syntax errors. Existing syntax error recovery techniques, like panic mode and multi-option repairs, are either too coarse, e.g. in deleting large swathes of code, or lead to a proliferation of possible completions. This paper introduces $\texttt{tall}~\texttt{tylr}$, an error-handling parser and editor generator that completes malformed code with $\textit{syntactic obligations}$ that abstract over many possible completions. These obligations generalize the familiar notion of holes in structure editors to cover missing operands, operators, delimiters, and sort transitions. $\texttt{tall}~\texttt{tylr}$ is backed by a novel theory of tile-based parsing, conceptually organized around a $\textit{molder}$ that turns tokens into tiles and a $\textit{melder}$ that completes and parses tiles into terms using an error-handling generalization of operator-precedence parsing. We formalize melding as a parsing calculus, $\textsf{meldr}$, that completes input tiles with additional obligations such that it can be parsed into a well-formed term, with success guaranteed over all inputs. We further describe how $\texttt{tall}~\texttt{tylr}$ implements molding and completion-ranking using the principle of $\textit{minimizing obligations}$. Obligations offer a useful way to scaffold internal program representations, but in $\texttt{tall}~\texttt{tylr}$ we go further to investigate the potential of $\textit{materializing}$ these obligations visually to the programmer. We conduct a user study to evaluate the extent to which an editor like $\texttt{tall}~\texttt{tylr}$ that materializes syntactic obligations might be usable and useful, finding both points of positivity and interesting new avenues for future work.