Time as a Control Dimension in Robot Learning

TL;DR

This paper introduces a novel reinforcement learning framework that treats time as a controllable dimension, enabling robots to adapt their behavior dynamically across various tasks and improve robustness and human controllability.

Contribution

The paper presents time-aware policy learning, a new approach that incorporates temporal signals into policies, allowing flexible and adaptive robot behavior across different temporal regimes.

Findings

Policies adapt behavior from rapid to deliberate actions.

Improves robustness under sim-to-real transfer and disturbances.

Enhances controllability with human input without retraining.

Abstract

Temporal awareness plays a central role in intelligent behavior by shaping how actions are paced, coordinated, and adapted to changing goals and environments. In contrast, most robot learning algorithms treat time only as a fixed episode horizon or scheduling constraint. Here we introduce time-aware policy learning, a reinforcement learning framework that treats time as a control dimension of robot behavior. The approach augments policies with two temporal signals, the remaining time and a time ratio that modulates the policy's internal progression of time, allowing a single policy to regulate its execution strategy across temporal regimes. Across diverse manipulation tasks including long-horizon manipulation, granular-media pouring, articulated-object interaction, and multi-agent coordination, the resulting policies adapt their behavior continuously from dynamic execution under tight schedules to stable and deliberate interaction when more time is available. This temporal awareness improves efficiency, robustness under sim-to-real mismatch and disturbances, and controllability through human input without retraining. Treating time as a controllable variable provides a new framework for adaptive and human-aligned robot autonomy.