Assistive Image Annotation Systems with Deep Learning and Natural Language Capabilities: A Review

TL;DR

This review discusses AI-assisted image annotation systems that use deep learning and natural language processing to improve annotation efficiency and quality across various computer vision tasks, highlighting current methods, datasets, and future directions.

Contribution

It provides a comprehensive overview of existing AI-assisted image annotation systems with textual output, emphasizing recent deep learning and natural language techniques, and identifies gaps for future research.

Findings

Limited publicly available work on AI-assisted annotation with text output

Deep learning methods enable semantic understanding and free-text generation

Future research needs more datasets and collaboration

Abstract

While supervised learning has achieved significant success in computer vision tasks, acquiring high-quality annotated data remains a bottleneck. This paper explores both scholarly and non-scholarly works in AI-assistive deep learning image annotation systems that provide textual suggestions, captions, or descriptions of the input image to the annotator. This potentially results in higher annotation efficiency and quality. Our exploration covers annotation for a range of computer vision tasks including image classification, object detection, regression, instance, semantic segmentation, and pose estimation. We review various datasets and how they contribute to the training and evaluation of AI-assistive annotation systems. We also examine methods leveraging neuro-symbolic learning, deep active learning, and self-supervised learning algorithms that enable semantic image understanding and generate free-text output. These include image captioning, visual question answering, and multi-modal reasoning. Despite the promising potential, there is limited publicly available work on AI-assistive image annotation with textual output capabilities. We conclude by suggesting future research directions to advance this field, emphasizing the need for more publicly accessible datasets and collaborative efforts between academia and industry.