Trinary Tools for Continuously Valued Binary Classifiers

TL;DR

This paper introduces an interactive visualization system for analyzing continuously-valued binary classifiers by treating their scores as trinary, aiding calibration, operating point selection, and examination tasks.

Contribution

It extends existing comparison-based visualization methods to continuous classifiers by modeling scores as trinary, supporting comprehensive classifier analysis tasks.

Findings

Enhanced visualization views for classifier analysis

Effective support for calibration and threshold selection

Demonstrated usability through practical use cases

Abstract

Classification methods for binary (yes/no) tasks often produce a continuously valued score. Machine learning practitioners must perform model selection, calibration, discretization, performance assessment, tuning, and fairness assessment. Such tasks involve examining classifier results, typically using summary statistics and manual examination of details. In this paper, we provide an interactive visualization approach to support such continuously-valued classifier examination tasks. Our approach addresses the three phases of these tasks: calibration, operating point selection, and examination. We enhance standard views and introduce task-specific views so that they can be integrated into a multi-view coordination (MVC) system. We build on an existing comparison-based approach, extending it to continuous classifiers by treating the continuous values as trinary (positive, unsure, negative) even if the classifier will not ultimately use the 3-way classification. We provide use cases that demonstrate how our approach enables machine learning practitioners to accomplish key tasks.