Hybrid Repeat/Multi-point Sampling for Highly Volatile Objective Functions

TL;DR

This paper introduces Hybrid Repeat/Multi-point Sampling, a novel technique to improve Bayesian optimization for highly volatile objective functions, enabling AI agents to better learn and adapt in uncertain environments.

Contribution

The paper proposes a new sampling method that enhances Gaussian Process surrogate accuracy and reliability in volatile settings, improving optimization and adaptability.

Findings

Enhanced Gaussian Process models for volatile functions

Improved optimization reliability in uncertain environments

Better behavioral adaptation for AI agents

Abstract

A key drawback of the current generation of artificial decision-makers is that they do not adapt well to changes in unexpected situations. This paper addresses the situation in which an AI for aerial dog fighting, with tunable parameters that govern its behavior, will optimize behavior with respect to an objective function that must be evaluated and learned through simulations. Once this objective function has been modeled, the agent can then choose its desired behavior in different situations. Bayesian optimization with a Gaussian Process surrogate is used as the method for investigating the objective function. One key benefit is that during optimization the Gaussian Process learns a global estimate of the true objective function, with predicted outcomes and a statistical measure of confidence in areas that haven't been investigated yet. However, standard Bayesian optimization does not perform consistently or provide an accurate Gaussian Process surrogate function for highly volatile objective functions. We treat these problems by introducing a novel sampling technique called Hybrid Repeat/Multi-point Sampling. This technique gives the AI ability to learn optimum behaviors in a highly uncertain environment. More importantly, it not only improves the reliability of the optimization, but also creates a better model of the entire objective surface. With this improved model the agent is equipped to better adapt behaviors.