# Latency-Aware Benchmarking of Large Language Models for Natural-Language Robot Navigation in ROS 2

## TL;DR

This paper introduces a benchmarking framework for testing how well large language models enable robot navigation using natural language, within the ROS 2 system.

## Contribution

The first reproducible multi-LLM system with multi-planner evaluations for natural-language robot navigation in ROS 2.

## Key findings

- Smaller LLMs respond faster but have less spatial reasoning, while larger models are more accurate but slower.
- The framework measures latency, accuracy, path quality, and task success in standard indoor scenarios.
- Different local planners (DWB, TEB, RPP) show varied performance when integrated with LLMs.

## Abstract

A growing challenge in mobile robotics is the reliance on complex graphical interfaces and rigid control pipelines, which limit accessibility for non-expert users. This work introduces a latency-aware benchmarking framework that enables natural-language robot navigation by integrating multiple Large Language Models (LLMs) with the Robot Operating System 2 (ROS 2) Navigation 2 (Nav2) stack. The system allows robots to interpret and act upon free-form text instructions, replacing traditional Human–Machine Interfaces (HMIs) with conversational interaction. Using a simulated TurtleBot4 platform in Gazebo Fortress, we benchmarked a diverse set of contemporary LLMs, including GPT-3.5, GPT-4, GPT-5, Claude 3.7, Gemini 2.5, Mistral-7B Instruct, DeepSeek-R1, and LLaMA-3.3-70B, across three local planners, namely Dynamic Window Approach (DWB), Timed Elastic Band (TEB), and Regulated Pure Pursuit (RPP). The framework measures end-to-end response latency, instruction-parsing accuracy, path quality, and task success rate in standardised indoor scenarios. The results show that there are clear trade-offs between latency and accuracy, where smaller models respond quickly but have less spatial reasoning, while larger models have more consistent navigation intent but take longer to respond. The proposed framework is the first reproducible multi-LLM system with multi-planner evaluations within ROS 2, supporting the development of intuitive and latency-efficient natural-language interfaces for robot navigation.