Social-R1: Towards Human-like Social Reasoning in LLMs

TL;DR

This paper introduces Social-R1, a reinforcement learning framework that enhances social reasoning in large language models by training with challenging cases and aligning reasoning processes with human cognition, leading to improved social intelligence.

Contribution

It presents a novel RL approach with multi-dimensional rewards and an adversarial benchmark to train models for human-like social reasoning.

Findings

A 4B parameter model surpasses larger models in social reasoning tasks.

The approach generalizes well across eight diverse benchmarks.

Trajectory-level alignment improves social intelligence in LLMs.

Abstract

While large language models demonstrate remarkable capabilities across numerous domains, social intelligence - the capacity to perceive social cues, infer mental states, and generate appropriate responses - remains a critical challenge, particularly for enabling effective human-AI collaboration and developing AI that truly serves human needs. Current models often rely on superficial patterns rather than genuine social reasoning. We argue that cultivating human-like social intelligence requires training with challenging cases that resist shortcut solutions. To this end, we introduce ToMBench-Hard, an adversarial benchmark designed to provide hard training examples for social reasoning. Building on this, we propose Social-R1, a reinforcement learning framework that aligns model reasoning with human cognition through multi-dimensional rewards. Unlike outcome-based RL, Social-R1 supervises the entire reasoning process, enforcing structural alignment, logical integrity, and information density. Results show that our approach enables a 4B parameter model to surpass much larger counterparts and generalize robustly across eight diverse benchmarks. These findings demonstrate that challenging training cases with trajectory-level alignment offer a path toward efficient and reliable social intelligence.