How Can LLMs and Knowledge Graphs Contribute to Robot Safety? A Few-Shot Learning Approach

TL;DR

This paper proposes a safety verification layer using Few-Shot learning and knowledge graph prompting to ensure safe execution of LLM-generated code in robotic systems, demonstrated on drone control in simulation.

Contribution

It introduces a novel safety layer combining fine-tuned GPT-4o with knowledge graph prompting to verify LLM-generated code for robotics.

Findings

Enhanced safety and compliance in drone control

Effective verification of LLM-generated code

Improved adherence to drone operation regulations

Abstract

Large Language Models (LLMs) are transforming the robotics domain by enabling robots to comprehend and execute natural language instructions. The cornerstone benefits of LLM include processing textual data from technical manuals, instructions, academic papers, and user queries based on the knowledge provided. However, deploying LLM-generated code in robotic systems without safety verification poses significant risks. This paper outlines a safety layer that verifies the code generated by ChatGPT before executing it to control a drone in a simulated environment. The safety layer consists of a fine-tuned GPT-4o model using Few-Shot learning, supported by knowledge graph prompting (KGP). Our approach improves the safety and compliance of robotic actions, ensuring that they adhere to the regulations of drone operations.