SHADE: Semantic Hypernym Annotator for Domain-specific Entities -- DnD Domain Use Case
Akila Peiris, Nisansa de Silva

TL;DR
SHADE is an annotation tool designed to improve the consistency and efficiency of annotating domain-specific entities, such as those in fantasy literature, by reducing human errors and standardizing labels.
Contribution
The paper introduces SHADE, a specialized annotation software tailored for the high fantasy domain, addressing challenges of consistency and error reduction in domain-specific data annotation.
Findings
Enhanced annotation consistency in fantasy domain data
Reduced human errors during entity annotation
Facilitated domain-specific entity labeling
Abstract
Manual data annotation is an important NLP task but one that takes considerable amount of resources and effort. In spite of the costs, labeling and categorizing entities is essential for NLP tasks such as semantic evaluation. Even though annotation can be done by non-experts in most cases, due to the fact that this requires human labor, the process is costly. Another major challenge encountered in data annotation is maintaining the annotation consistency. Annotation efforts are typically carried out by teams of multiple annotators. The annotations need to maintain the consistency in relation to both the domain truth and annotation format while reducing human errors. Annotating a specialized domain that deviates significantly from the general domain, such as fantasy literature, will see a lot of human error and annotator disagreement. So it is vital that proper guidelines and error…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
