Dual Embodied-Symbolic Concept Representations for Deep Learning

TL;DR

This paper proposes a dual-level concept representation model combining embodied feature vectors and symbolic knowledge graphs, aiming to enhance deep learning and AI integration through two key applications.

Contribution

It introduces a novel dual embodied-symbolic concept representation framework inspired by cognitive science, enabling improved few-shot learning and multimodal understanding.

Findings

Effective knowledge distillation for few-shot learning

Enhanced image-text matching accuracy

Foundation for deep learning and symbolic AI integration

Abstract

Motivated by recent findings from cognitive neural science, we advocate the use of a dual-level model for concept representations: the embodied level consists of concept-oriented feature representations, and the symbolic level consists of concept graphs. Embodied concept representations are modality specific and exist in the form of feature vectors in a feature space. Symbolic concept representations, on the other hand, are amodal and language specific, and exist in the form of word / knowledge-graph embeddings in a concept / knowledge space. The human conceptual system comprises both embodied representations and symbolic representations, which typically interact to drive conceptual processing. As such, we further advocate the use of dual embodied-symbolic concept representations for deep learning. To demonstrate their usage and value, we discuss two important use cases: embodied-symbolic knowledge distillation for few-shot class incremental learning, and embodied-symbolic fused representation for image-text matching. Dual embodied-symbolic concept representations are the foundation for deep learning and symbolic AI integration. We discuss two important examples of such integration: scene graph generation with knowledge graph bridging, and multimodal knowledge graphs.