Cognitively-Inspired Emergent Communication via Knowledge Graphs for Assisting the Visually Impaired

TL;DR

This paper introduces VAG-EC, a cognitively-inspired emergent communication framework using knowledge graphs to improve real-time assistive navigation for the visually impaired, balancing speed and semantic richness.

Contribution

It presents a novel knowledge graph-based emergent communication method that emulates human cognition, enhancing interpretability and context-awareness in assistive systems.

Findings

VAG-EC outperforms traditional methods in TopSim and CI metrics.

The framework effectively balances latency and semantic depth.

Experiments show improved interpretability and adaptability.

Abstract

Assistive systems for visually impaired individuals must deliver rapid, interpretable, and adaptive feedback to facilitate real-time navigation. Current approaches face a trade-off between latency and semantic richness: natural language-based systems provide detailed guidance but are too slow for dynamic scenarios, while emergent communication frameworks offer low-latency symbolic languages but lack semantic depth, limiting their utility in tactile modalities like vibration. To address these limitations, we introduce a novel framework, Cognitively-Inspired Emergent Communication via Knowledge Graphs (VAG-EC), which emulates human visual perception and cognitive mapping. Our method constructs knowledge graphs to represent objects and their relationships, incorporating attention mechanisms to prioritize task-relevant entities, thereby mirroring human selective attention. This structured approach enables the emergence of compact, interpretable, and context-sensitive symbolic languages. Extensive experiments across varying vocabulary sizes and message lengths demonstrate that VAG-EC outperforms traditional emergent communication methods in Topographic Similarity (TopSim) and Context Independence (CI). These findings underscore the potential of cognitively grounded emergent communication as a fast, adaptive, and human-aligned solution for real-time assistive technologies. Code is available at https://github.com/Anonymous-NLPcode/Anonymous_submission/tree/main.