CoRefi: A Crowd Sourcing Suite for Coreference Annotation

TL;DR

CoRefi is a web-based, open-source platform designed to facilitate efficient and cost-effective coreference annotation through crowdsourcing, featuring guided onboarding and a review algorithm.

Contribution

It introduces CoRefi, a novel crowdsourcing suite with integrated onboarding and review features for coreference annotation, enhancing accessibility and efficiency.

Findings

CoRefi reduces annotation time and cost.

The review algorithm improves annotation quality.

Open source integration enables easy deployment.

Abstract

Coreference annotation is an important, yet expensive and time consuming, task, which often involved expert annotators trained on complex decision guidelines. To enable cheaper and more efficient annotation, we present CoRefi, a web-based coreference annotation suite, oriented for crowdsourcing. Beyond the core coreference annotation tool, CoRefi provides guided onboarding for the task as well as a novel algorithm for a reviewing phase. CoRefi is open source and directly embeds into any website, including popular crowdsourcing platforms. CoRefi Demo: aka.ms/corefi Video Tour: aka.ms/corefivideo Github Repo: https://github.com/aribornstein/corefi