Fast, Flexible, and Declarative Construction of Abstract Syntax Trees with PEGs

TL;DR

This paper introduces a declarative method for constructing abstract syntax trees using PEGs, featuring new AST operators and a transactional AST machine to ensure consistency during parsing, implemented in the Nez parser.

Contribution

It proposes a novel declarative approach with AST operators and a transactional AST machine to improve AST construction in PEG-based parsers.

Findings

Transactional AST machine adds ~25% parsing time

Automated consistency control for ASTs

Implemented in Java Nez parser

Abstract

We address a declarative construction of abstract syntax trees with Parsing Expression Grammars. AST operators (constructor, connector, and tagging) are newly defined to specify flexible AST constructions. A new challenge coming with PEGs is the consistency management of ASTs in backtracking and packrat parsing. We make the transaction AST machine in order to perform AST operations in the context of the speculative parsing of PEGs. All the consistency control is automated by the analysis of AST operators. The proposed approach is implemented in the Nez parser, written in Java. The performance study shows that the transactional AST machine requires 25\% approximately more time in CSV, XML, and C grammars.