Quantifying Sensitivity for Tree Ensembles: A symbolic and compositional approach

TL;DR

This paper introduces a novel, scalable method for quantifying the sensitivity of decision tree ensembles by discretizing input space and using algebraic decision diagrams, enabling efficient verification in safety-critical applications.

Contribution

The authors develop a new algorithm that encodes sensitivity analysis as algebraic decision diagrams, making the process scalable and providing certified error bounds.

Findings

XCount achieves significant speedup over existing methods.

The approach scales well with increasing ensemble size.

Experimental results validate the efficiency and scalability of the method.

Abstract

Decision tree ensembles (DTE) are a popular model for a wide range of AI classification tasks, used in multiple safety critical domains, and hence verifying properties on these models has been an active topic of study over the last decade. One such verification question is the problem of sensitivity, which asks, given a DTE, whether a small change in subset of features can lead to misclassification of the input. In this work, our focus is to build a quantitative notion of sensitivity, tailored to DTEs, by discretizing the input space of the model and enumerating the regions which are susceptible to sensitivity. We propose a novel algorithmic technique that can perform this computation efficiently, within a certified error and confidence bound. Our approach is based on encoding the problem as an algebraic decision diagram (ADD), and further splitting it into subproblems that can be solved efficiently and make the computation compositional and scalable. We evaluate the performance of our technique over benchmarks of varying size in terms of number of trees and depth, comparing it against the performance of model counters over the same problem encoding. Experimental results show that our tool XCount achieves significant speedup over other approaches and can scale well with the increasing sizes of the ensembles.