Autoware.Flex: Human-Instructed Dynamically Reconfigurable Autonomous Driving Systems

TL;DR

Autoware.Flex introduces a human-instructed autonomous driving system that uses natural language translation and validation mechanisms to improve decision-making safety and incorporate user preferences, demonstrated through simulator and real-world tests.

Contribution

The paper presents a novel ADS framework that integrates human instructions via LLMs and validation to enhance safety and personalization in autonomous driving.

Findings

Effective interpretation of human instructions in simulation and real-world

Improved decision safety through validation mechanisms

Enhanced user satisfaction by incorporating preferences

Abstract

Existing Autonomous Driving Systems (ADS) independently make driving decisions, but they face two significant limitations. First, in complex scenarios, ADS may misinterpret the environment and make inappropriate driving decisions. Second, these systems are unable to incorporate human driving preferences in their decision-making processes. This paper proposes Autoware$.$Flex, a novel ADS system that incorporates human input into the driving process, allowing users to guide the ADS in making more appropriate decisions and ensuring their preferences are satisfied. Achieving this needs to address two key challenges: (1) translating human instructions, expressed in natural language, into a format the ADS can understand, and (2) ensuring these instructions are executed safely and consistently within the ADS' s decision-making framework. For the first challenge, we employ a Large Language Model (LLM) assisted by an ADS-specialized knowledge base to enhance domain-specific translation. For the second challenge, we design a validation mechanism to ensure that human instructions result in safe and consistent driving behavior. Experiments conducted on both simulators and a real-world autonomous vehicle demonstrate that Autoware$.$Flex effectively interprets human instructions and executes them safely.