SocialAI 0.1: Towards a Benchmark to Stimulate Research on Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

TL;DR

This paper introduces SocialAI 0.1, a benchmark designed to evaluate and stimulate research on complex socio-cognitive abilities in deep reinforcement learning agents, emphasizing diverse social interactions and multimodal communication.

Contribution

It proposes a new benchmark environment for testing social skills in Deep RL agents, integrating insights from cognitive sciences to guide future AI development.

Findings

Current SOTA Deep RL approaches have limitations in complex social scenarios.

The SocialAI benchmark reveals gaps in agents' social and multimodal communication skills.

The paper provides a foundation for future research on human-like social intelligence in AI.

Abstract

Building embodied autonomous agents capable of participating in social interactions with humans is one of the main challenges in AI. This problem motivated many research directions on embodied language use. Current approaches focus on language as a communication tool in very simplified and non diverse social situations: the "naturalness" of language is reduced to the concept of high vocabulary size and variability. In this paper, we argue that aiming towards human-level AI requires a broader set of key social skills: 1) language use in complex and variable social contexts; 2) beyond language, complex embodied communication in multimodal settings within constantly evolving social worlds. In this work we explain how concepts from cognitive sciences could help AI to draw a roadmap towards human-like intelligence, with a focus on its social dimensions. We then study the limits of a recent SOTA Deep RL approach when tested on a first grid-world environment from the upcoming SocialAI, a benchmark to assess the social skills of Deep RL agents. Videos and code are available at https://sites.google.com/view/socialai01 .