ModelWizard: Toward Interactive Model Construction

TL;DR

ModelWizard introduces an interactive, composable framework for model construction that allows data scientists to explore and refine machine learning models more naturally and efficiently.

Contribution

It presents a novel interactive model construction framework with primitive and composite operations, implemented as a domain-specific language in F# for tabular data.

Findings

Prototype demonstrates flexible model creation

Enables exploration of multiple candidate models

Facilitates understanding and validation of models

Abstract

Data scientists engage in model construction to discover machine learning models that well explain a dataset, in terms of predictiveness, understandability and generalization across domains. Questions such as "what if we model common cause Z" and "what if Y's dependence on X reverses" inspire many candidate models to consider and compare, yet current tools emphasize constructing a final model all at once. To more naturally reflect exploration when debating numerous models, we propose an interactive model construction framework grounded in composable operations. Primitive operations capture core steps refining data and model that, when verified, form an inductive basis to prove model validity. Derived, composite operations enable advanced model families, both generic and specialized, abstracted away from low-level details. We prototype our envisioned framework in ModelWizard, a domain-specific language embedded in F# to construct Tabular models. We enumerate language design and demonstrate its use through several applications, emphasizing how language may facilitate creation of complex models. To future engineers designing data science languages and tools, we offer ModelWizard's design as a new model construction paradigm, speeding discovery of our universe's structure.