Symbol emergence as interpersonal cross-situational learning: the emergence of lexical knowledge with combinatoriality

TL;DR

This paper introduces a computational model enabling agents to develop lexical knowledge with combinatoriality through cross-situational learning and semiotic communication, demonstrating generalization to new situations in simulated humanoid robots.

Contribution

The model uniquely integrates category formation from multimodal sensory-motor data with semiotic communication, advancing symbol emergence research.

Findings

Agents successfully acquired lexical knowledge with combinatoriality.

The model demonstrated generalization to novel situations.

Interpersonal cross-situational learning was effective in simulated experiments.

Abstract

We present a computational model for a symbol emergence system that enables the emergence of lexical knowledge with combinatoriality among agents through a Metropolis-Hastings naming game and cross-situational learning. Many computational models have been proposed to investigate combinatoriality in emergent communication and symbol emergence in cognitive and developmental robotics. However, existing models do not sufficiently address category formation based on sensory-motor information and semiotic communication through the exchange of word sequences within a single integrated model. Our proposed model facilitates the emergence of lexical knowledge with combinatoriality by performing category formation using multimodal sensory-motor information and enabling semiotic communication through the exchange of word sequences among agents in a unified model. Furthermore, the model enables an agent to predict sensory-motor information for unobserved situations by combining words associated with categories in each modality. We conducted two experiments with two humanoid robots in a simulated environment to evaluate our proposed model. The results demonstrated that the agents can acquire lexical knowledge with combinatoriality through interpersonal cross-situational learning based on the Metropolis-Hastings naming game and cross-situational learning. Furthermore, our results indicate that the lexical knowledge developed using our proposed model exhibits generalization performance for novel situations through interpersonal cross-modal inference.