Selecting Diverse Models for Scientific Insight

TL;DR

This paper introduces a multi-model penalized regression approach to identify diverse models that reveal multiple explanatory patterns in data, addressing model uncertainty in scientific modeling.

Contribution

It proposes a novel method for selecting multiple diverse models simultaneously, explicitly limiting model similarity to uncover different underlying predictor patterns.

Findings

Successfully identified multiple models explaining SFE from steel composition.

Demonstrated the method's ability to promote model diversity and variable selection.

Showed potential for revealing varied scientific insights from data.

Abstract

Model selection often aims to choose a single model, assuming that the form of the model is correct. However, there may be multiple possible underlying explanatory patterns in a set of predictors that could explain a response. Model selection without regard for model uncertainty can fail to bring these patterns to light. We explore multi-model penalized regression (MMPR) to acknowledge model uncertainty in the context of penalized regression. We examine how different penalty settings can promote either shrinkage or sparsity of coefficients in separate models. The method is tuned to explicitly limit model similarity. A choice of penalty form that enforces variable selection is applied to predict stacking fault energy (SFE) from steel alloy composition. The aim is to identify multiple models with different subsets of covariates that explain a single type of response.