Probe-Based Interventions for Modifying Agent Behavior

TL;DR

This paper introduces a method for modifying pre-trained neural networks to align their behavior with human-specified properties, enhancing human-agent collaboration across various AI systems.

Contribution

It presents a novel approach inspired by explainability techniques to update neural network representations based on external guidance, enabling behavior modification without retraining.

Findings

Improved human-agent team performance in experiments.

Effective modification of neural networks from image classifiers to reinforcement learning agents.

Demonstrated applicability across different neural network architectures.

Abstract

Neural nets are powerful function approximators, but the behavior of a given neural net, once trained, cannot be easily modified. We wish, however, for people to be able to influence neural agents' actions despite the agents never training with humans, which we formalize as a human-assisted decision-making problem. Inspired by prior art initially developed for model explainability, we develop a method for updating representations in pre-trained neural nets according to externally-specified properties. In experiments, we show how our method may be used to improve human-agent team performance for a variety of neural networks from image classifiers to agents in multi-agent reinforcement learning settings.