It's Not Just Labeling -- A Research on LLM Generated Feedback Interpretability and Image Labeling Sketch Features

TL;DR

This paper presents a sketch-based annotation method supported by large language models to improve image labeling accessibility, interpretability, and scalability, especially for non-experts, by analyzing sketch features and feedback quality.

Contribution

It introduces a novel sketch-based virtual assistant that enhances LLM-assisted labeling, focusing on interpretability, accessibility, and scalability for non-expert users.

Findings

Sketch recognition features correlate with LLM feedback metrics.

Prompting strategies and sketch variations affect feedback quality.

The approach improves annotation accessibility for non-experts.

Abstract

The quality of training data is critical to the performance of machine learning applications in domains like transportation, healthcare, and robotics. Accurate image labeling, however, often relies on time-consuming, expert-driven methods with limited feedback. This research introduces a sketch-based annotation approach supported by large language models (LLMs) to reduce technical barriers and enhance accessibility. Using a synthetic dataset, we examine how sketch recognition features relate to LLM feedback metrics, aiming to improve the reliability and interpretability of LLM-assisted labeling. We also explore how prompting strategies and sketch variations influence feedback quality. Our main contribution is a sketch-based virtual assistant that simplifies annotation for non-experts and advances LLM-driven labeling tools in terms of scalability, accessibility, and explainability.