Reducing numerical precision preserves classification accuracy in Mondrian Forests

TL;DR

Reducing floating-point precision in Mondrian Forests significantly decreases memory usage without harming classification accuracy, and can sometimes enhance performance due to regularization effects.

Contribution

This study demonstrates that lowering floating-point precision in Mondrian Forests from 64 to 8 bits reduces memory needs while maintaining or improving accuracy, highlighting a new hyperparameter for optimization.

Findings

Precision reduction from 64 to 8 bits retains F1 score.

In some cases, reduced precision improves classification performance.

Floating-point precision acts as a hyperparameter influencing accuracy.

Abstract

Mondrian Forests are a powerful data stream classification method, but their large memory footprint makes them ill-suited for low-resource platforms such as connected objects. We explored using reduced-precision floating-point representations to lower memory consumption and evaluated its effect on classification performance. We applied the Mondrian Forest implementation provided by OrpailleCC, a C++ collection of data stream algorithms, to two canonical datasets in human activity recognition: Recofit and Banos \emph{et al}. Results show that the precision of floating-point values used by tree nodes can be reduced from 64 bits to 8 bits with no significant difference in F1 score. In some cases, reduced precision was shown to improve classification performance, presumably due to its regularization effect. We conclude that numerical precision is a relevant hyperparameter in the Mondrian Forest, and that commonly-used double precision values may not be necessary for optimal performance. Future work will evaluate the generalizability of these findings to other data stream classifiers.