Scanning and Parsing Languages with Ambiguities and Constraints: The Lamb and Fence Algorithms

TL;DR

This paper introduces Lamb and Fence, two algorithms that enable flexible language processing by supporting ambiguities and constraints, thus decoupling language design from processing.

Contribution

The paper presents Lamb and Fence algorithms that handle ambiguities and constraints, facilitating model-based language processing beyond traditional grammar constraints.

Findings

Lamb supports ambiguous token definitions and custom matchers.

Fence handles ambiguous context-free grammars with constraints.

Together, they enable flexible, model-based language processing.

Abstract

Traditional language processing tools constrain language designers to specific kinds of grammars. In contrast, model-based language processing tools decouple language design from language processing. These tools allow the occurrence of lexical and syntactic ambiguities in language specifications and the declarative specification of constraints for resolving them. As a result, these techniques require scanners and parsers able to parse context-free grammars, handle ambiguities, and enforce constraints for disambiguation. In this paper, we present Lamb and Fence. Lamb is a scanning algorithm that supports ambiguous token definitions and the specification of custom pattern matchers and constraints. Fence is a chart parsing algorithm that supports ambiguous context-free grammars and the definition of constraints on associativity, composition, and precedence, as well as custom constraints. Lamb and Fence, in conjunction, enable the implementation of the ModelCC model-based language processing tool.