Visually Analyzing and Steering Zero Shot Learning

TL;DR

This paper introduces a visual analytics system that enables users to analyze, diagnose, and improve zero-shot learning models by understanding their failures and guiding modifications.

Contribution

It presents a novel visualization tool specifically designed for diagnosing and steering zero-shot learning models, addressing a key challenge in understanding model failures.

Findings

Helps identify failure points in zero-shot models

Enables users to understand causes of mispredictions

Assists in improving zero-shot learning performance

Abstract

We propose a visual analytics system to help a user analyze and steer zero-shot learning models. Zero-shot learning has emerged as a viable scenario for categorizing data that consists of no labeled examples, and thus a promising approach to minimize data annotation from humans. However, it is challenging to understand where zero-shot learning fails, the cause of such failures, and how a user can modify the model to prevent such failures. Our visualization system is designed to help users diagnose and understand mispredictions in such models, so that they may gain insight on the behavior of a model when applied to data associated with categories not seen during training. Through usage scenarios, we highlight how our system can help a user improve performance in zero-shot learning.