# Giving AI agents a sense of control facilitates reinforcement learning in multitasking scenarios

## TL;DR

This paper explores how giving AI agents a sense of control improves their ability to handle multiple tasks at once, similar to how humans manage complex situations.

## Contribution

The novel contribution is a mathematical model of a situational sense of control integrated into reinforcement learning agents.

## Key findings

- Agents with a sense of control showed significant performance improvements in multitasking scenarios.
- These agents prioritized complex tasks and switched between them at optimal times.
- The model outperformed agents without a sense of control and matched human-like behavior.

## Abstract

Having to control multiple tasks in parallel poses challenges for humans and artificial agents alike. In artificial intelligence, specific forms of reinforcement learning (RL), most notably hierarchical and model-based RL, have shown promising results in scenarios where tasks or skills need to be switched adaptively. However, RL agents still encounter difficulties when faced with serial multitasking that involves switching control between continuously running subtasks, such as changing the radio station while driving in traffic. Inspired by human cognitive processes, we hypothesize that maintaining a sense of control is a key mechanism facilitating such task-switching decisions. We propose a mathematical formulation of a situational sense of control that consists of two components: an evaluative indicator of the predictability of action outcomes and a predictive indicator of a need for control in individual subtasks. We integrate this model of a sense of control into a hierarchical RL agent and evaluate its performance in a Collect Asteroids game environment, in which one must alternate between navigating two spaceships to collect as many asteroids as possible. Comparing RL agents with and without a sense of control, as well as with human participants, shows that equipping RL agents with a sense of control results in significant performance improvements. Our findings indicate that agents equipped with a sense of control prioritize more complex tasks, exhibit increased switching behavior, and make switches at strategically optimal times, leading to superior overall performance. The incorporation of cognitive mechanisms, inspired by human behavior, into RL agents thus appears to yield considerable enhancements in performance when acting in complex and dynamic environments.