Enhancing Model Assessment in Vision-based Interactive Machine Teaching through Real-time Saliency Map Visualization

TL;DR

This paper introduces a vision-based interactive machine teaching interface that uses real-time saliency map visualization to improve user understanding of model behavior during assessment, leading to better model training.

Contribution

The paper presents a novel interactive teaching system with real-time saliency visualization, enhancing model assessment beyond confidence scores.

Findings

Saliency maps provide clearer insights into model focus areas.

Real-time visualization helps users correct model concepts more effectively.

Improved model training accuracy through better user guidance.

Abstract

Interactive Machine Teaching systems allow users to create customized machine learning models through an iterative process of user-guided training and model assessment. They primarily offer confidence scores of each label or class as feedback for assessment by users. However, we observe that such feedback does not necessarily suffice for users to confirm the behavior of the model. In particular, confidence scores do not always offer the full understanding of what features in the data are used for learning, potentially leading to the creation of an incorrectly-trained model. In this demonstration paper, we present a vision-based interactive machine teaching interface with real-time saliency map visualization in the assessment phase. This visualization can offer feedback on which regions of each image frame the current model utilizes for classification, thus better guiding users to correct the corresponding concepts in the iterative teaching.