Accurator: Nichesourcing for Cultural Heritage

TL;DR

Accurator introduces a nichesourcing methodology and a specialized annotation tool to gather high-quality, domain-specific metadata for cultural heritage objects by engaging niche communities through tailored campaigns.

Contribution

The paper presents a novel nichesourcing methodology, a web-based annotation tool for experts, and validates their effectiveness across three diverse cultural heritage domains.

Findings

High-quality annotations were successfully collected in all case studies.

The methodology and tool improved annotation quality and quantity.

Users found the tool usable for domain-specific tasks.

Abstract

With more and more cultural heritage data being published online, their usefulness in this open context depends on the quality and diversity of descriptive metadata for collection objects. In many cases, existing metadata is not adequate for a variety of retrieval and research tasks and more specific annotations are necessary. However, eliciting such annotations is a challenge since it often requires domain-specific knowledge. Where crowdsourcing can be successfully used for eliciting simple annotations, identifying people with the required expertise might prove troublesome for tasks requiring more complex or domain-specific knowledge. Nichesourcing addresses this problem, by tapping into the expert knowledge available in niche communities. This paper presents Accurator, a methodology for conducting nichesourcing campaigns for cultural heritage institutions, by addressing communities, organizing events and tailoring a web-based annotation tool to a domain of choice. The contribution of this paper is threefold: 1) a nichesourcing methodology, 2) an annotation tool for experts and 3) validation of the methodology and tool in three case studies. The three domains of the case studies are birds on art, bible prints and fashion images. We compare the quality and quantity of obtained annotations in the three case studies, showing that the nichesourcing methodology in combination with the image annotation tool can be used to collect high quality annotations in a variety of domains and annotation tasks. A user evaluation indicates the tool is suited and usable for domain specific annotation tasks.