PEACE: Prompt Engineering Automation for CLIPSeg Enhancement for Safe-Landing Zone Segmentation

TL;DR

PEACE automates prompt engineering for CLIPSeg to improve safe landing zone segmentation in robotics, dynamically adapting to environmental changes and significantly increasing safe landing identification accuracy.

Contribution

It introduces a novel system that automatically generates and refines prompts for safe landing zone segmentation, enhancing robustness over traditional fixed-prompt methods.

Findings

Successful safe landing zone identification from 57% to 92% accuracy.

Outperforms standard CLIP and CLIPSeg prompting methods.

Enhanced performance with FastSAM replacement.

Abstract

Safe landing is essential in robotics applications, from industrial settings to space exploration. As artificial intelligence advances, we have developed PEACE (Prompt Engineering Automation for CLIPSeg Enhancement), a system that automatically generates and refines prompts for identifying landing zones in changing environments. Traditional approaches using fixed prompts for open-vocabulary models struggle with environmental changes and can lead to dangerous outcomes when conditions are not represented in the predefined prompts. PEACE addresses this limitation by dynamically adapting to shifting data distributions. Our key innovation is the dual segmentation of safe and unsafe landing zones, allowing the system to refine the results by removing unsafe areas from potential landing sites. Using only monocular cameras and image segmentation, PEACE can safely guide descent operations from 100 meters to altitudes as low as 20 meters. The testing shows that PEACE significantly outperforms the standard CLIP and CLIPSeg prompting methods, improving the successful identification of safe landing zones from 57% to 92%. We have also demonstrated enhanced performance when replacing CLIPSeg with FastSAM. The complete source code is available as an open-source software.