OpEvo: An Evolutionary Method for Tensor Operator Optimization

TL;DR

OpEvo is an evolutionary algorithm that efficiently optimizes tensor operator configurations for deep neural networks by leveraging topological structures, outperforming state-of-the-art methods in search efficiency and results.

Contribution

The paper introduces OpEvo, a novel topology-aware evolutionary method for tensor operator optimization that improves search efficiency and effectiveness over existing approaches.

Findings

OpEvo finds optimal configurations with fewer trials.

It achieves lower variance in results.

It reduces computational effort compared to SOTA methods.

Abstract

Training and inference efficiency of deep neural networks highly rely on the performance of tensor operators on hardware platforms. Manually optimizing tensor operators has limitations in terms of supporting new operators or hardware platforms. Therefore, automatically optimizing device code configurations of tensor operators is getting increasingly attractive. However, current methods for tensor operator optimization usually suffer from poor sample-efficiency due to the combinatorial search space. In this work, we propose a novel evolutionary method, OpEvo, which efficiently explores the search spaces of tensor operators by introducing a topology-aware mutation operation based on q-random walk to leverage the topological structures over the search spaces. Our comprehensive experiment results show that compared with state-of-the-art (SOTA) methods OpEvo can find the best configuration with the lowest variance and least efforts in the number of trials and wall-clock time. All code of this work is available online.