Modulation of temporal decision-making in a deep reinforcement learning agent under the dual-task paradigm

TL;DR

This paper investigates how dual-task interference affects temporal decision-making in deep reinforcement learning agents, revealing overproduction of time in dual tasks and exploring neural dynamics without clear timer evidence.

Contribution

It demonstrates emergent human-like timing behavior in DRL agents under dual-task conditions and highlights the need for further analysis of underlying neural mechanisms.

Findings

Dual-task agents overproduce time relative to single-task agents.

Behavioral patterns are consistent across multiple target durations.

Preliminary neural analysis shows no clear dedicated timer mechanism.

Abstract

This study explores the interference in temporal processing within a dual-task paradigm from an artificial intelligence (AI) perspective. In this context, the dual-task setup is implemented as a simplified version of the Overcooked environment with two variations, single task (T) and dual task (T+N). Both variations involve an embedded time production task, but the dual task (T+N) additionally involves a concurrent number comparison task. Two deep reinforcement learning (DRL) agents were separately trained for each of these tasks. These agents exhibited emergent behavior consistent with human timing research. Specifically, the dual task (T+N) agent exhibited significant overproduction of time relative to its single task (T) counterpart. This result was consistent across four target durations. Preliminary analysis of neural dynamics in the agents' LSTM layers did not reveal any clear evidence of a dedicated or intrinsic timer. Hence, further investigation is needed to better understand the underlying time-keeping mechanisms of the agents and to provide insights into the observed behavioral patterns. This study is a small step towards exploring parallels between emergent DRL behavior and behavior observed in biological systems in order to facilitate a better understanding of both.