Tisane: Authoring Statistical Models via Formal Reasoning from Conceptual and Data Relationships

TL;DR

Tisane is a system that helps data analysts formally specify, reason about, and construct valid generalized linear models by capturing domain knowledge and data relationships, reducing errors and improving scientific validity.

Contribution

Tisane introduces a study design language and an interactive compilation process for authoring statistical models based on domain and data relationships.

Findings

Researchers found Tisane helps focus on goals and assumptions.

Tisane reduces modeling mistakes in case studies.

System supports disambiguation and valid model construction.

Abstract

Proper statistical modeling incorporates domain theory about how concepts relate and details of how data were measured. However, data analysts currently lack tool support for recording and reasoning about domain assumptions, data collection, and modeling choices in an integrated manner, leading to mistakes that can compromise scientific validity. For instance, generalized linear mixed-effects models (GLMMs) help answer complex research questions, but omitting random effects impairs the generalizability of results. To address this need, we present Tisane, a mixed-initiative system for authoring generalized linear models with and without mixed-effects. Tisane introduces a study design specification language for expressing and asking questions about relationships between variables. Tisane contributes an interactive compilation process that represents relationships in a graph, infers candidate statistical models, and asks follow-up questions to disambiguate user queries to construct a valid model. In case studies with three researchers, we find that Tisane helps them focus on their goals and assumptions while avoiding past mistakes.