Autotuning Apache TVM-based Scientific Applications Using Bayesian Optimization

TL;DR

This paper introduces a Bayesian Optimization-based autotuning framework for Apache TVM to enhance the performance of scientific linear algebra kernels on GPUs, outperforming existing autotuning methods.

Contribution

It presents a novel autotuning framework using Bayesian Optimization for TVM, specifically targeting dense matrix factorizations on GPUs, with demonstrated superior performance.

Findings

Our framework outperforms AutoTVM in most cases.

Significant performance improvements on GPU clusters.

Effective tuning of linear algebra kernels using Bayesian methods.

Abstract

Apache TVM (Tensor Virtual Machine), an open source machine learning compiler framework designed to optimize computations across various hardware platforms, provides an opportunity to improve the performance of dense matrix factorizations such as LU (Lower Upper) decomposition and Cholesky decomposition on GPUs and AI (Artificial Intelligence) accelerators. In this paper, we propose a new TVM autotuning framework using Bayesian Optimization and use the TVM tensor expression language to implement linear algebra kernels such as LU, Cholesky, and 3mm. We use these scientific computation kernels to evaluate the effectiveness of our methods on a GPU cluster, called Swing, at Argonne National Laboratory. We compare the proposed autotuning framework with the TVM autotuning framework AutoTVM with four tuners and find that our framework outperforms AutoTVM in most cases.