FLInt: Exploiting Floating Point Enabled Integer Arithmetic for Efficient Random Forest Inference

TL;DR

This paper introduces FLInt, a method that replaces floating point comparisons with integer operations in random forest inference, reducing overhead and improving performance without sacrificing accuracy.

Contribution

The paper presents FLInt, a novel integer-based comparison method for floating point numbers in random forests, formally proven correct and hardware-efficient.

Findings

Reduces inference time by up to 30% on various systems.

Eliminates the need for floating point hardware in random forest inference.

Maintains model accuracy while improving performance.

Abstract

In many machine learning applications, e.g., tree-based ensembles, floating point numbers are extensively utilized due to their expressiveness. Nowadays performing data analysis on embedded devices from dynamic data masses becomes available, but such systems often lack hardware capabilities to process floating point numbers, introducing large overheads for their processing. Even if such hardware is present in general computing systems, using integer operations instead of floating point operations promises to reduce operation overheads and improve the performance. In this paper, we provide \mdname, a full precision floating point comparison for random forests, by only using integer and logic operations. To ensure the same functionality preserves, we formally prove the correctness of this comparison. Since random forests only require comparison of floating point numbers during inference, we implement \mdname~in low level realizations and therefore eliminate the need for floating point hardware entirely, by keeping the model accuracy unchanged. The usage of \mdname~basically boils down to a one-by-one replacement of conditions: For instance, a comparison statement in C: if(pX[3]<=(float)10.074347) becomes if((*(((int*)(pX))+3))<=((int)(0x41213087))). Experimental evaluation on X86 and ARMv8 desktop and server class systems shows that the execution time can be reduced by up to $\approx 30\%$ with our novel approach.