Towards Computationally Verifiable Semantic Grounding for Language Models

TL;DR

This paper introduces a novel method for semantic grounding of language models by integrating a semantic parser within an auto-encoder framework, improving the fluency and semantic accuracy of generated text.

Contribution

It proposes two techniques—candidate sampling and training with a frozen semantic parser—to enhance semantic grounding in language models.

Findings

Significant improvement over greedy search baseline in fluency and semantic accuracy.

Human evaluation confirms automatic metrics results.

Effective use of entity-relationship triples for semantic representation.

Abstract

The paper presents an approach to semantic grounding of language models (LMs) that conceptualizes the LM as a conditional model generating text given a desired semantic message formalized as a set of entity-relationship triples. It embeds the LM in an auto-encoder by feeding its output to a semantic parser whose output is in the same representation domain as the input message. Compared to a baseline that generates text using greedy search, we demonstrate two techniques that improve the fluency and semantic accuracy of the generated text: The first technique samples multiple candidate text sequences from which the semantic parser chooses. The second trains the language model while keeping the semantic parser frozen to improve the semantic accuracy of the auto-encoder. We carry out experiments on the English WebNLG 3.0 data set, using BLEU to measure the fluency of generated text and standard parsing metrics to measure semantic accuracy. We show that our proposed approaches significantly improve on the greedy search baseline. Human evaluation corroborates the results of the automatic evaluation experiments.