Evaluating Few-Shot Pill Recognition Under Visual Domain Shift

TL;DR

This paper evaluates the effectiveness of few-shot learning for pill recognition in complex, real-world scenarios, emphasizing generalization across domain shifts and the importance of realistic training data.

Contribution

It demonstrates that few-shot fine-tuning enables rapid adaptation in pill recognition, highlighting the significance of training data realism for deployment robustness.

Findings

Semantic recognition adapts quickly with few examples.

Localization performance declines under occlusion.

Realistic training data improves robustness in low-shot settings.

Abstract

Adverse drug events are a significant source of preventable harm, which has led to the development of automated pill recognition systems to enhance medication safety. Real-world deployment of these systems is hindered by visually complex conditions, including cluttered scenes, overlapping pills, reflections, and diverse acquisition environments. This study investigates few-shot pill recognition from a deployment-oriented perspective, prioritizing generalization under realistic cross-dataset domain shifts over architectural innovation. A two-stage object detection framework is employed, involving base training followed by few-shot fine-tuning. Models are adapted to novel pill classes using one, five, or ten labeled examples per class and are evaluated on a separate deployment dataset featuring multi-object, cluttered scenes. The evaluation focuses on classification-centric and error-based metrics to address heterogeneous annotation strategies. Findings indicate that semantic pill recognition adapts rapidly with few-shot supervision, with classification performance reaching saturation even with a single labeled example. However, stress testing under overlapping and occluded conditions demonstrates a marked decline in localization and recall, despite robust semantic classification. Models trained on visually realistic, multi-pill data consistently exhibit greater robustness in low-shot scenarios, underscoring the importance of training data realism and the diagnostic utility of few-shot fine-tuning for deployment readiness.