Neural Network Surgery with Sets

TL;DR

This paper introduces a set-based approach for neural network surgery, enabling automatic identification of which parts of a model to salvage or retrain after architecture or feature set modifications, demonstrated on OpenAI Five.

Contribution

The paper proposes a novel set-based methodology for neural network surgery that automates the process of determining salvageable components after architecture changes.

Findings

Successfully transferred weights across feature and architecture modifications in OpenAI Five.

Derived two methods for detecting feature-parameter interaction maps and proved their equivalence.

Validated the approach empirically on the OpenAI Five model.

Abstract

The cost to train machine learning models has been increasing exponentially, making exploration and research into the correct features and architecture a costly or intractable endeavor at scale. However, using a technique named "surgery" OpenAI Five was continuously trained to play the game DotA 2 over the course of 10 months through 20 major changes in features and architecture. Surgery transfers trained weights from one network to another after a selection process to determine which sections of the model are unchanged and which must be re-initialized. In the past, the selection process relied on heuristics, manual labor, or pre-existing boundaries in the structure of the model, limiting the ability to salvage experiments after modifications of the feature set or input reorderings. We propose a solution to automatically determine which components of a neural network model should be salvaged and which require retraining. We achieve this by allowing the model to operate over discrete sets of features and use set-based operations to determine the exact relationship between inputs and outputs, and how they change across tweaks in model architecture. In this paper, we introduce the methodology for enabling neural networks to operate on sets, derive two methods for detecting feature-parameter interaction maps, and show their equivalence. We empirically validate that we can surgery weights across feature and architecture changes to the OpenAI Five model.