From Modal to Multimodal Ambiguities: a Classification Approach

TL;DR

This paper introduces a new classification of multimodal ambiguities, distinguishing semantic and syntactic types, and demonstrates a rule-based method achieving over 92% accuracy in classifying these ambiguities.

Contribution

It develops an original, empirically validated classification system for multimodal ambiguities from a linguistic perspective, with a rule-based classification method.

Findings

Achieved 94.6% accuracy for semantic ambiguity classification.

Achieved 92.1% accuracy for syntactic ambiguity classification.

Validated the classification method against human judgment.

Abstract

This paper deals with classifying ambiguities for Multimodal Languages. It evolves the classifications and the methods of the literature on ambiguities for Natural Language and Visual Language, empirically defining an original classification of ambiguities for multimodal interaction using a linguistic perspective. This classification distinguishes between Semantic and Syntactic multimodal ambiguities and their subclasses, which are intercepted using a rule-based method implemented in a software module. The experimental results have achieved an accuracy of the obtained classification compared to the expected one, which are defined by the human judgment, of 94.6% for the semantic ambiguities classes, and 92.1% for the syntactic ambiguities classes.