Towards Adaptive Training of Agent-based Sparring Partners for Fighter Pilots

TL;DR

This paper presents a novel Bayesian optimization approach with Hybrid Repeat/Multi-point Sampling to train adaptive AI agents for aerial dogfighting, improving reliability and understanding of volatile decision spaces.

Contribution

It introduces a new sampling technique that enhances Bayesian optimization for volatile environments, enabling better training of adaptive agent-based sparring partners.

Findings

Improved optimization reliability in uncertain environments

Enhanced Gaussian Process surrogate accuracy

Better prediction of agent performance in unseen scenarios

Abstract

A key requirement for the current generation of artificial decision-makers is that they should adapt well to changes in unexpected situations. This paper addresses the situation in which an AI for aerial dog fighting, with tunable parameters that govern its behavior, must optimize behavior with respect to an objective function that is evaluated and learned through simulations. Bayesian optimization with a Gaussian Process surrogate is used as the method for investigating the objective function. One key benefit is that during optimization, the Gaussian Process learns a global estimate of the true objective function, with predicted outcomes and a statistical measure of confidence in areas that haven't been investigated yet. Having a model of the objective function is important for being able to understand possible outcomes in the decision space; for example this is crucial for training and providing feedback to human pilots. However, standard Bayesian optimization does not perform consistently or provide an accurate Gaussian Process surrogate function for highly volatile objective functions. We treat these problems by introducing a novel sampling technique called Hybrid Repeat/Multi-point Sampling. This technique gives the AI ability to learn optimum behaviors in a highly uncertain environment. More importantly, it not only improves the reliability of the optimization, but also creates a better model of the entire objective surface. With this improved model the agent is equipped to more accurately/efficiently predict performance in unexplored scenarios.