Lexical State Analyzer

TL;DR

The paper introduces a novel analysis tool, LSA, that detects errors and warnings in JavaCC grammars with lexical states, providing visualizations and counterexamples to improve grammar correctness.

Contribution

It presents a new context sensitive and insensitive analysis for lexical states in CFGs, along with a practical tool for JavaCC grammar validation.

Findings

Effective identification of errors and warnings in JavaCC grammars.

Visualization of grammar errors through graphs.

Generation of counterexamples for error analysis.

Abstract

Lexical states provide a powerful mechanism to scan regular expressions in a context sensitive manner. At the same time, lexical states also make it hard to reason about the correctness of the grammar. We first categorize the related correctness issues into two classes: errors and warnings, and then present a context sensitive and a context insensitive analysis to identify errors and warnings in context-free-grammars (CFGs). We also present a comparative study of these analyses. A standalone tool (LSA) has also been implemented by us that can identify errors and warnings in JavaCC grammars. The LSA tool outputs a graph that depicts the grammar and the error transitions. It can also generates counter example strings that can be used to establish the errors. We have used LSA to analyze a host of open-source JavaCC grammar files to good effect.