Packrat Parsing: Simple, Powerful, Lazy, Linear Time

TL;DR

Packrat parsing is a simple, powerful, and linear-time parsing technique that combines the flexibility of backtracking with the efficiency of linear algorithms, enabling easier handling of complex grammars in functional programming.

Contribution

This paper introduces packrat parsing, a novel approach that achieves linear-time parsing with backtracking and unlimited lookahead, expanding the capabilities of traditional top-down parsers.

Findings

Recognizes any LL(k) or LR(k) grammar

Simplifies handling of syntactic idioms like longest-match rule

Enables integration of lexical analysis into parsing

Abstract

Packrat parsing is a novel technique for implementing parsers in a lazy functional programming language. A packrat parser provides the power and flexibility of top-down parsing with backtracking and unlimited lookahead, but nevertheless guarantees linear parse time. Any language defined by an LL(k) or LR(k) grammar can be recognized by a packrat parser, in addition to many languages that conventional linear-time algorithms do not support. This additional power simplifies the handling of common syntactic idioms such as the widespread but troublesome longest-match rule, enables the use of sophisticated disambiguation strategies such as syntactic and semantic predicates, provides better grammar composition properties, and allows lexical analysis to be integrated seamlessly into parsing. Yet despite its power, packrat parsing shares the same simplicity and elegance as recursive descent parsing; in fact converting a backtracking recursive descent parser into a linear-time packrat parser often involves only a fairly straightforward structural change. This paper describes packrat parsing informally with emphasis on its use in practical applications, and explores its advantages and disadvantages with respect to the more conventional alternatives.