To tune or not to tune the number of trees in random forest?

TL;DR

This paper investigates how the number of trees in a random forest affects performance, revealing that error rates can be non-monotonous and suggesting setting the number based on convergence rather than maximum size.

Contribution

It provides theoretical insights into the non-monotonous behavior of error rates with respect to the number of trees and offers practical guidelines for choosing this number.

Findings

Expected error rate may decrease then increase with more trees.

Other performance measures like Brier score do not show non-monotonous patterns.

Large-scale dataset analysis supports setting a large, computationally feasible number of trees.

Abstract

The number of trees T in the random forest (RF) algorithm for supervised learning has to be set by the user. It is controversial whether T should simply be set to the largest computationally manageable value or whether a smaller T may in some cases be better. While the principle underlying bagging is that "more trees are better", in practice the classification error rate sometimes reaches a minimum before increasing again for increasing number of trees. The goal of this paper is four-fold: (i) providing theoretical results showing that the expected error rate may be a non-monotonous function of the number of trees and explaining under which circumstances this happens; (ii) providing theoretical results showing that such non-monotonous patterns cannot be observed for other performance measures such as the Brier score and the logarithmic loss (for classification) and the mean squared error (for regression); (iii) illustrating the extent of the problem through an application to a large number (n = 306) of datasets from the public database OpenML; (iv) finally arguing in favor of setting it to a computationally feasible large number, depending on convergence properties of the desired performance measure.