Human Guided Learning of Transparent Regression Models

TL;DR

This paper introduces HuGuR, a human-in-the-loop method for transparent permutation regression models that allows users to interactively add constraints, enabling understandable models that perform well on small datasets.

Contribution

The paper presents HuGuR, a novel interactive approach for building transparent regression models with human-understandable constraints, evaluated through user studies and benchmarked against baselines.

Findings

User-built models outperform baselines on small datasets.

Models are interpretable and understandable to humans.

Machine models outperform user models on larger datasets.

Abstract

We present a human-in-the-loop (HIL) approach to permutation regression, the novel task of predicting a continuous value for a given ordering of items. The model is a gradient boosted regression model that incorporates simple human-understandable constraints of the form x < y, i.e. item x has to be before item y, as binary features. The approach, HuGuR (Human Guided Regression), lets a human explore the search space of such transparent regression models. Interacting with HuGuR, users can add, remove, and refine order constraints interactively, while the coefficients are calculated on the fly. We evaluate HuGuR in a user study and compare the performance of user-built models with multiple baselines on 9 data sets. The results show that the user-built models outperform the compared methods on small data sets and in general perform on par with the other methods, while being in principle understandable for humans. On larger datasets from the same domain, machine-induced models begin to outperform the user-built models. Further work will study the trust users have in models when constructed by themselves and how the scheme can be transferred to other pattern domains, such as strings, sequences, trees, or graphs.