FamilySeer: Towards Optimized Tensor Codes by Exploiting Computation Subgraph Similarity

TL;DR

FamilySeer is an auto-tuning framework for deep learning compilers that leverages subgraph similarities to improve tensor code optimization efficiency and quality within limited time budgets.

Contribution

It introduces a novel approach to organize subgraphs into families, enabling shared tuning and more accurate cost models to enhance auto-tuning performance.

Findings

Achieves better tensor code optimization with limited tuning time.

Outperforms state-of-the-art auto-tuning frameworks in efficiency.

Uses subgraph family organization to improve cost model accuracy.

Abstract

Deploying various deep learning (DL) models efficiently has boosted the research on DL compilers. The difficulty of generating optimized tensor codes drives DL compiler to ask for the auto-tuning approaches, and the increasing demands require increasing auto-tuning efficiency and quality. Currently, the DL compilers partition the input DL models into several subgraphs and leverage the auto-tuning to find the optimal tensor codes of these subgraphs. However, existing auto-tuning approaches usually regard subgraphs as individual ones and overlook the similarities across them, and thus fail to exploit better tensor codes under limited time budgets. We propose FamilySeer, an auto-tuning framework for DL compilers that can generate better tensor codes even with limited time budgets. FamilySeer exploits the similarities and differences among subgraphs can organize them into subgraph families, where the tuning of one subgraph can also improve other subgraphs within the same family. The cost model of each family gets more purified training samples generated by the family and becomes more accurate so that the costly measurements on real hardware can be replaced with the lightweight estimation through cost model. Our experiments show that FamilySeer can generate model codes with the same code performance more efficiently than state-of-the-art auto-tuning frameworks.