Retrieval from Captioned Image Databases Using Natural Language Processing

TL;DR

This paper presents ANVIL, a natural language processing system that improves image retrieval accuracy from captioned databases by analyzing query-caption relations and providing contextual understanding, applicable in research and commercial settings.

Contribution

It introduces a novel natural language technique for image retrieval that leverages caption analysis and contextual information to enhance accuracy and user understanding.

Findings

High accuracy retrieval of captioned images achieved

Natural language techniques enable contextual understanding

System successfully used in research and commercial applications

Abstract

It might appear that natural language processing should improve the accuracy of information retrieval systems, by making available a more detailed analysis of queries and documents. Although past results appear to show that this is not so, if the focus is shifted to short phrases rather than full documents, the situation becomes somewhat different. The ANVIL system uses a natural language technique to obtain high accuracy retrieval of images which have been annotated with a descriptive textual caption. The natural language techniques also allow additional contextual information to be derived from the relation between the query and the caption, which can help users to understand the overall collection of retrieval results. The techniques have been successfully used in a information retrieval system which forms both a testbed for research and the basis of a commercial system.