Advanced LPeg techniques: A dual case study approach

TL;DR

This paper introduces advanced optimization techniques for Lua Parsing Expression Grammars (LPeg), significantly enhancing JSON parsing speed and pattern conversion efficiency through strategic grammar construction and implementation of novel search strategies.

Contribution

It presents novel optimization strategies for LPeg, including substitution capture, segment-boundary separation, and flattened search, improving parsing performance without modifying the LPeg library.

Findings

JSON parser achieves up to 125 MB/s processing speed

Glob-to-LPeg converter outperforms Bun.Glob by 14-92%

Parser runs 3-14 times faster than Minimatch

Abstract

This paper presents advanced optimization techniques for Lua Parsing Expression Grammars (LPeg) through two complementary case studies: a high-performance JSON parser and a sophisticated Glob-to-LPeg pattern converter. We demonstrate how strategic grammar construction can dramatically improve parsing performance without modifying the underlying LPeg library. For the JSON parser, we implement substitution capture and table construction optimization to reduce memory allocation overhead and improve object processing. For the Glob converter, we introduce segment-boundary separation, implement Cox's flattened search strategy, and develop optimized braced condition handling to prevent exponential backtracking. Comprehensive benchmarks demonstrate that our JSON parser achieves processing speeds up to 125 MB/s on complex documents, consistently outperforming dkjson and showing competitive results against rxi_json across most test cases. Our Glob-to-LPeg converter exhibits 14-92% better performance than Bun.Glob and runs 3-14 times faster than Minimatch across diverse pattern matching scenarios. This research provides practical optimization techniques for LPeg-based parsers, contributing valuable strategies to the text processing ecosystem.