Variable selection and sensitivity analysis using dynamic trees, with an application to computer code performance tuning

TL;DR

This paper introduces dynamic tree-based tools for variable selection and sensitivity analysis tailored to nonlinear, noisy, and non-stationary response functions, with applications in optimizing computer code performance.

Contribution

It presents a novel set of methods using dynamic trees for variable selection and sensitivity analysis, specifically designed for complex, real-world computer code tuning problems.

Findings

Faster analysis compared to static tree methods

Richer feature sets for variable importance

Effective in code-tuning and error detection

Abstract

We investigate an application in the automatic tuning of computer codes, an area of research that has come to prominence alongside the recent rise of distributed scientific processing and heterogeneity in high-performance computing environments. Here, the response function is nonlinear and noisy and may not be smooth or stationary. Clearly needed are variable selection, decomposition of influence, and analysis of main and secondary effects for both real-valued and binary inputs and outputs. Our contribution is a novel set of tools for variable selection and sensitivity analysis based on the recently proposed dynamic tree model. We argue that this approach is uniquely well suited to the demands of our motivating example. In illustrations on benchmark data sets, we show that the new techniques are faster and offer richer feature sets than do similar approaches in the static tree and computer experiment literature. We apply the methods in code-tuning optimization, examination of a cold-cache effect, and detection of transformation errors.