# Robot gains Social Intelligence through Multimodal Deep Reinforcement Learning

## TL;DR

This paper introduces a Multimodal Deep Q-Network enabling robots to learn human-like social interaction skills through trial-and-error reinforcement learning from high-dimensional sensory data.

## Contribution

It presents a novel end-to-end reinforcement learning approach for robots to acquire social skills via multimodal sensory inputs.

## Key findings

- Robot learned basic social interaction skills within 14 days
- Demonstrated successful learning from high-dimensional sensory data
- Showed potential for autonomous social skill acquisition

## Abstract

For robots to coexist with humans in a social world like ours, it is crucial that they possess human-like social interaction skills. Programming a robot to possess such skills is a challenging task. In this paper, we propose a Multimodal Deep Q-Network (MDQN) to enable a robot to learn human-like interaction skills through a trial and error method. This paper aims to develop a robot that gathers data during its interaction with a human and learns human interaction behaviour from the high-dimensional sensory information using end-to-end reinforcement learning. This paper demonstrates that the robot was able to learn basic interaction skills successfully, after 14 days of interacting with people.