LLM-Enhanced Rapid-Reflex Async-Reflect Embodied Agent for Real-Time Decision-Making in Dynamically Changing Environments

TL;DR

This paper introduces RRARA, a latency-aware embodied agent leveraging LLMs and a novel Time Conversion Mechanism to improve real-time decision-making in high-risk, dynamically changing environments.

Contribution

It presents RRARA, a new asynchronous agent with a latency-aware evaluation protocol and a Time Conversion Mechanism for better real-time decisions in extreme scenarios.

Findings

RRARA outperforms existing baselines in latency-sensitive environments.

The Time Conversion Mechanism effectively aligns decision delays with simulation frames.

Latency-aware evaluation improves understanding of agent performance in dynamic settings.

Abstract

In the realm of embodied intelligence, the evolution of large language models (LLMs) has markedly enhanced agent decision making. Consequently, researchers have begun exploring agent performance in dynamically changing high-risk scenarios, i.e., fire, flood, and wind scenarios in the HAZARD benchmark. Under these extreme conditions, the delay in decision making emerges as a crucial yet insufficiently studied issue. We propose a Time Conversion Mechanism (TCM) that translates inference delays in decision-making into equivalent simulation frames, thus aligning cognitive and physical costs under a single FPS-based metric. By extending HAZARD with Respond Latency (RL) and Latency-to-Action Ratio (LAR), we deliver a fully latency-aware evaluation protocol. Moreover, we present the Rapid-Reflex Async-Reflect Agent (RRARA), which couples a lightweight LLM-guided feedback module with a rule-based agent to enable immediate reactive behaviors and asynchronous reflective refinements in situ. Experiments on HAZARD show that RRARA substantially outperforms existing baselines in latency-sensitive scenarios.