Vectorization of Gradient Boosting of Decision Trees Prediction in the CatBoost Library for RISC-V Processors

TL;DR

This paper explores RISC-V-specific vectorization of the CatBoost library to optimize decision tree predictions, demonstrating significant speedups through manual intrinsics-based vectorization on RISC-V processors.

Contribution

It introduces a RISC-V-focused vectorization approach for CatBoost, enabling efficient utilization of RISC-V CPU resources through manual optimization techniques.

Findings

Vectorization with intrinsics speeds up CatBoost predictions on RISC-V.

Manual optimization achieves several times faster decision tree computations.

Open-source code is available for further research and development.

Abstract

The emergence and rapid development of the open RISC-V instruction set architecture opens up new horizons on the way to efficient devices, ranging from existing low-power IoT boards to future high-performance servers. The effective use of RISC-V CPUs requires software optimization for the target platform. In this paper, we focus on the RISC-V-specific optimization of the CatBoost library, one of the widely used implementations of gradient boosting for decision trees. The CatBoost library is deeply optimized for commodity CPUs and GPUs. However, vectorization is required to effectively utilize the resources of RISC-V CPUs with the RVV 0.7.1 vector extension, which cannot be done automatically with a C++ compiler yet. The paper reports on our experience in benchmarking CatBoost on the Lichee Pi 4a, RISC-V-based board, and shows how manual vectorization of computationally intensive loops with intrinsics can speed up the use of decision trees several times, depending on the specific workload. The developed codes are publicly available on GitHub.