CMLCompiler: A Unified Compiler for Classical Machine Learning

TL;DR

CMLCompiler is a unified compiler framework that optimizes classical machine learning inference across various hardware, achieving significant speedups and improving performance and portability in hybrid ML deployments.

Contribution

It introduces a unified abstraction and conversion framework for CML inference, enabling optimized execution on diverse hardware through integration with DL compilers.

Findings

Up to 4.38× CPU speedup

Up to 3.31× GPU speedup

Up to 5.09× IoT device speedup

Abstract

Classical machine learning (CML) occupies nearly half of machine learning pipelines in production applications. Unfortunately, it fails to utilize the state-of-the-practice devices fully and performs poorly. Without a unified framework, the hybrid deployments of deep learning (DL) and CML also suffer from severe performance and portability issues. This paper presents the design of a unified compiler, called CMLCompiler, for CML inference. We propose two unified abstractions: operator representations and extended computational graphs. The CMLCompiler framework performs the conversion and graph optimization based on two unified abstractions, then outputs an optimized computational graph to DL compilers or frameworks. We implement CMLCompiler on TVM. The evaluation shows CMLCompiler's portability and superior performance. It achieves up to 4.38$\times$ speedup on CPU, 3.31$\times$ speedup on GPU, and 5.09$\times$ speedup on IoT devices, compared to the state-of-the-art solutions -- scikit-learn, intel sklearn, and hummingbird. Our performance of CML and DL mixed pipelines achieves up to 3.04x speedup compared with cross-framework implementations. The project documents and source code are available at https://www.computercouncil.org/cmlcompiler.