Embedding Grammars

TL;DR

This paper introduces hybrid semantic grammars that combine traditional context-free grammar structures with word embeddings, enabling more flexible and semantically aware pattern matching.

Contribution

It presents a novel approach to grammar design that integrates word embeddings, allowing grammars to generalize over semantically related words and phrases.

Findings

Enables grammars to match semantic regions in vector space

Reduces grammar complexity by generalizing terminals

Improves matching flexibility with semantic awareness

Abstract

Classic grammars and regular expressions can be used for a variety of purposes, including parsing, intent detection, and matching. However, the comparisons are performed at a structural level, with constituent elements (words or characters) matched exactly. Recent advances in word embeddings show that semantically related words share common features in a vector-space representation, suggesting the possibility of a hybrid grammar and word embedding. In this paper, we blend the structure of standard context-free grammars with the semantic generalization capabilities of word embeddings to create hybrid semantic grammars. These semantic grammars generalize the specific terminals used by the programmer to other words and phrases with related meanings, allowing the construction of compact grammars that match an entire region of the vector space rather than matching specific elements.