Academically intelligent LLMs are not necessarily socially intelligent
Ruoxi Xu, Hongyu Lin, Xianpei Han, Le Sun, Yingfei Sun

TL;DR
This study develops a standardized test to evaluate the social intelligence of large language models, revealing that their social skills are limited and distinct from their academic abilities, with social factors influencing their performance.
Contribution
The paper introduces SESI, a novel social intelligence assessment framework for LLMs, and provides extensive evaluation results highlighting the gap between academic and social intelligence.
Findings
LLMs show significant room for improvement in social intelligence.
Superficial friendliness is a primary source of errors.
Social intelligence in LLMs is influenced by social factors and is distinct from academic intelligence.
Abstract
The academic intelligence of large language models (LLMs) has made remarkable progress in recent times, but their social intelligence performance remains unclear. Inspired by established human social intelligence frameworks, particularly Daniel Goleman's social intelligence theory, we have developed a standardized social intelligence test based on real-world social scenarios to comprehensively assess the social intelligence of LLMs, termed as the Situational Evaluation of Social Intelligence (SESI). We conducted an extensive evaluation with 13 recent popular and state-of-art LLM agents on SESI. The results indicate the social intelligence of LLMs still has significant room for improvement, with superficially friendliness as a primary reason for errors. Moreover, there exists a relatively low correlation between the social intelligence and academic intelligence exhibited by LLMs,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLibrary Science and Information Systems · Digital Rights Management and Security · Wikis in Education and Collaboration
