MetaStackVis: Visually-Assisted Performance Evaluation of Metamodels

TL;DR

MetaStackVis is a visualization tool designed to evaluate and compare the performance of various metamodels in stacking ensemble methods, aiding users in selecting optimal configurations through interactive visual exploration.

Contribution

It introduces MetaStackVis, a novel visualization system for assessing different metamodels' impacts on stacking ensemble performance, extending previous work with a focus on multiple metamodels.

Findings

MetaStackVis enables effective visual comparison of metamodels.

The tool helps identify the best metamodels for specific data instances.

Expert feedback confirms its usefulness in performance evaluation.

Abstract

Stacking (or stacked generalization) is an ensemble learning method with one main distinctiveness from the rest: even though several base models are trained on the original data set, their predictions are further used as input data for one or more metamodels arranged in at least one extra layer. Composing a stack of models can produce high-performance outcomes, but it usually involves a trial-and-error process. Therefore, our previously developed visual analytics system, StackGenVis, was mainly designed to assist users in choosing a set of top-performing and diverse models by measuring their predictive performance. However, it only employs a single logistic regression metamodel. In this paper, we investigate the impact of alternative metamodels on the performance of stacking ensembles using a novel visualization tool, called MetaStackVis. Our interactive tool helps users to visually explore different singular and pairs of metamodels according to their predictive probabilities and multiple validation metrics, as well as their ability to predict specific problematic data instances. MetaStackVis was evaluated with a usage scenario based on a medical data set and via expert interviews.