A Metasemantic-Metapragmatic Framework for Taxonomizing Multimodal Communicative Alignment

TL;DR

This paper introduces a comprehensive metasemantic-metapragmatic framework based on Peircean categories to better understand and categorize multimodal human-like communication, emphasizing the importance of indexical contextualization.

Contribution

It proposes a novel taxonomy grounded in pragmatist philosophy, highlighting the role of indexical contextualization and directionality in multimodal communication, which is underrepresented in current methodologies.

Findings

Highlights the importance of indexicality in multimodal communication

Introduces the concept of indexical contextualization and directionality

Discusses implications for human-machine alignment and ethics

Abstract

Drawing on contemporary pragmatist philosophy and linguistic theories on cognition, meaning, and communication, this paper presents a dynamic, metasemantic-metapragmatic taxonomy for grounding and conceptualizing human-like multimodal communicative alignment. The framework is rooted in contemporary developments of the three basic communicative capacities initially identified by American logician and pragmatist philosopher Charles Sanders Peirce: iconic (sensory and perceptual qualities), indexical (contextual and sociocultural associations), and rule-like (symbolic and intuitive reasoning). Expanding on these developments, I introduce the concept of indexical contextualization and propose the principle of "contextualization directionality" for characterizing the crucial metapragmatic capacity for maintaining, navigating, or transitioning between semantic and pragmatic modes of multimodal communication. I contend that current cognitive-social computational and engineering methodologies disproportionately emphasize the semantic/metasemantic domain, overlooking the pivotal role of metapragmatic indexicality in traversing the semantic-pragmatic spectrum of communication. The framework's broader implications for intentionality, identity, affect, and ethics in within-modal and cross-modal human-machine alignment are also discussed.