NACSOS-nexus: NLP Assisted Classification, Synthesis and Online Screening with New and EXtended Usage Scenarios

TL;DR

NACSOS is a web platform that streamlines evidence synthesis by supporting data curation, annotation management, prioritised screening, and statistical stopping, facilitating systematic mapping and review processes.

Contribution

It introduces a comprehensive, web-based ecosystem with advanced annotation, screening, and quality control features, including the first implementation of statistical stopping criteria for systematic maps.

Findings

Supports prioritised screening algorithms

Enables resolution of multi-coder annotations with quality metrics

Provides tools for creating living maps and curated corpora

Abstract

NACSOS is a web-based platform for curating data used in systematic maps. It contains several (experimental) features that aid the evidence synthesis process from finding and ingesting primary data (mainly scientific publications), basic search and exploration thereof, but mainly the handling of managing the manual and automated annotations. The platform supports prioritised screening algorithms and is the first to fully implement statistical stopping criteria. Annotations by multiple coders can be resolved and customisable quality metrics are computed on-the-fly. In its current state, the annotations are performed on document level. The ecosystem around NACSOS offers packages for accessing the underlying database and practical utility functions that have proven useful in a multitude of projects. Further, it provides the backbone of living maps, review ecosystems, and our public literature hub for sharing high-quality curated corpora.