Measuring semantic complexity

TL;DR

This paper introduces a novel way to quantify semantic complexity in natural language understanding using meaning automata, enabling measurement of system complexity prior to development.

Contribution

It proposes a new concept of meaning automata to measure semantic complexity and demonstrates its application to various natural language understanding systems.

Findings

Semantic complexity can be quantified before system implementation.

Complex behaviors can emerge from simple semantic components.

The approach applies to understanding prepositional phrases and language interfaces.

Abstract

We define {\em semantic complexity} using a new concept of {\em meaning automata}. We measure the semantic complexity of understanding of prepositional phrases, of an "in depth understanding system", and of a natural language interface to an on-line calendar. We argue that it is possible to measure some semantic complexities of natural language processing systems before building them, and that systems that exhibit relatively complex behavior can be built from semantically simple components.