Meta-active Learning in Probabilistically-Safe Optimization

TL;DR

This paper introduces a meta-active learning method using an LSTM-based acquisition function for safe, efficient control of complex systems, demonstrating significant improvements in information gain and safety in simulations and real-world applications.

Contribution

It presents a novel meta-learning approach for safe active learning with an LSTM-based acquisition function, optimized via mixed-integer linear programming for high-dimensional control tasks.

Findings

Achieved 46% increase in information gain over baselines.

Reduced computation time by 20%.

Increased information gain by 58% in deep brain stimulation.

Abstract

Learning to control a safety-critical system with latent dynamics (e.g. for deep brain stimulation) requires taking calculated risks to gain information as efficiently as possible. To address this problem, we present a probabilistically-safe, meta-active learning approach to efficiently learn system dynamics and optimal configurations. We cast this problem as meta-learning an acquisition function, which is represented by a Long-Short Term Memory Network (LSTM) encoding sampling history. This acquisition function is meta-learned offline to learn high quality sampling strategies. We employ a mixed-integer linear program as our policy with the final, linearized layers of our LSTM acquisition function directly encoded into the objective to trade off expected information gain (e.g., improvement in the accuracy of the model of system dynamics) with the likelihood of safe control. We set a new state-of-the-art in active learning for control of a high-dimensional system with altered dynamics (i.e., a damaged aircraft), achieving a 46% increase in information gain and a 20% speedup in computation time over baselines. Furthermore, we demonstrate our system's ability to learn the optimal parameter settings for deep brain stimulation in a rat's brain while avoiding unwanted side effects (i.e., triggering seizures), outperforming prior state-of-the-art approaches with a 58% increase in information gain. Additionally, our algorithm achieves a 97% likelihood of terminating in a safe state while losing only 15% of information gain.