Simulation-based Scenario Generation for Robust Hybrid AI for Autonomy

TL;DR

This paper introduces HAMERITT, a simulation framework that generates mission-relevant scenarios with semantic context to train and test hybrid neural-symbolic AI algorithms for UAV autonomy, enhancing reasoning capabilities in complex environments.

Contribution

Hameritt provides a novel simulation environment with scenario generation that includes symbolic context, supporting hybrid AI training and testing for UAV autonomous operations.

Findings

Supports training of neuro-symbolic algorithms for UAVs

Includes scenario generation with symbolic and spatial-temporal context

Facilitates testing and assurance of hybrid AI in simulated missions

Abstract

Application of Unmanned Aerial Vehicles (UAVs) in search and rescue, emergency management, and law enforcement has gained traction with the advent of low-cost platforms and sensor payloads. The emergence of hybrid neural and symbolic AI approaches for complex reasoning is expected to further push the boundaries of these applications with decreasing levels of human intervention. However, current UAV simulation environments lack semantic context suited to this hybrid approach. To address this gap, HAMERITT (Hybrid Ai Mission Environment for RapId Training and Testing) provides a simulation-based autonomy software framework that supports the training, testing and assurance of neuro-symbolic algorithms for autonomous maneuver and perception reasoning. HAMERITT includes scenario generation capabilities that offer mission-relevant contextual symbolic information in addition to raw sensor data. Scenarios include symbolic descriptions for entities of interest and their relations to scene elements, as well as spatial-temporal constraints in the form of time-bounded areas of interest with prior probabilities and restricted zones within those areas. HAMERITT also features support for training distinct algorithm threads for maneuver vs. perception within an end-to-end mission run. Future work includes improving scenario realism and scaling symbolic context generation through automated workflow.