CheckSupport: A Local LLM-Powered Tool for Automated Manuscript Submission Checklist Selection and Completion

TL;DR

CheckSupport is a locally deployable LLM-based tool that automates manuscript reporting checklist recommendations and completions, enhancing transparency and reproducibility in scientific research.

Contribution

It introduces a staged prompting approach for local LLM inference, achieving high accuracy in checklist automation without compromising data privacy.

Findings

90% accuracy in checklist recommendations

88% accuracy in checklist item completion

12.5 seconds average processing time per manuscript

Abstract

Transparent and standardized reporting is essential for reproducible scientific research, yet adherence to reporting guidelines remains inconsistent because of the manual effort required to select and complete checklists. We present CheckSupport, an open-source, locally deployable system that uses large language models to automate the recommendation of reporting checklists and the evidence-grounded completion of checklists for scientific manuscripts. CheckSupport employs a staged prompting strategy that decomposes reporting workflows into constrained inference tasks, prioritizing faithful extraction over generative text synthesis. All inference is performed locally using instruction-tuned models, preserving data privacy and enabling reproducible, auditable workflows. Evaluated on a corpus of peer-reviewed manuscripts, CheckSupport achieved 90% overall accuracy for checklist recommendations and 88% overall accuracy for item-level completion while operating on CPU-only hardware. On average, the wall-clock time per manuscript was 12.5 seconds, including the checklist recommendation and full checklist completion. These results demonstrate that large language models, when applied as structured inference components, can reduce reporting burden and support more transparent and reproducible scientific reporting across disciplines.