Target-independent XLA optimization using Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning-based method to optimize the sequence of compiler passes in XLA independently of target hardware, leading to significant reductions in operation counts for neural network graphs.

Contribution

It proposes a novel RL-based approach for target-independent XLA pass ordering and develops an experimental framework for training and evaluating RL agents in this context.

Findings

13.3% operation count reduction on GPT-2 graphs

10.4% operation count reduction on diverse neural network graphs

Enhanced RL algorithms improve search performance

Abstract

An important challenge in Machine Learning compilers like XLA is multi-pass optimization and analysis. There has been recent interest chiefly in XLA target-dependent optimization on the graph-level, subgraph-level, and kernel-level phases. We specifically focus on target-independent optimization XLA HLO pass ordering: our approach aims at finding the optimal sequence of compiler optimization passes, which is decoupled from target-dependent optimization. However, there is little domain specific study in pass ordering for XLA HLO. To this end, we propose introducing deep Reinforcement Learning (RL) based search for optimal XLA HLO pass ordering. We also propose enhancements to the deep RL algorithms to further improve optimal search performance and open the research direction for domain-specific guidance for RL. We create an XLA Gym experimentation framework as a tool to enable RL algorithms to interact with the compiler for passing optimizations and thereby train agents. Overall, in our experimentation we observe an average of $13.3\%$ improvement in operation count reduction on a benchmark of GPT-2 training graphs and $10.4\%$ improvement on a diverse benchmark including GPT-2, BERT, and ResNet graphs using the proposed approach over the compiler's default phase ordering.