Reinforced Genetic Algorithm Learning for Optimizing Computation Graphs

TL;DR

This paper introduces a deep reinforcement learning method that trains offline to optimize neural network computation graphs, significantly reducing execution time and memory usage in real-world scenarios.

Contribution

It presents a novel offline training approach for reinforcement learning-based graph optimization that generalizes to unseen graphs, outperforming existing methods.

Findings

Achieves high-quality optimization decisions in seconds.

Outperforms classical and learning-based baselines.

Reduces execution time and peak memory usage.

Abstract

We present a deep reinforcement learning approach to minimizing the execution cost of neural network computation graphs in an optimizing compiler. Unlike earlier learning-based works that require training the optimizer on the same graph to be optimized, we propose a learning approach that trains an optimizer offline and then generalizes to previously unseen graphs without further training. This allows our approach to produce high-quality execution decisions on real-world TensorFlow graphs in seconds instead of hours. We consider two optimization tasks for computation graphs: minimizing running time and peak memory usage. In comparison to an extensive set of baselines, our approach achieves significant improvements over classical and other learning-based methods on these two tasks.