What Is Novel? A Knowledge-Driven Framework for Bias-Aware Literature Originality Evaluation

TL;DR

This paper presents a knowledge-driven framework that uses large language models to evaluate research novelty by comparing manuscripts to existing work, aligning with human judgment and improving consistency.

Contribution

It introduces a structured, knowledge-based approach for bias-aware novelty assessment that learns from peer reviews and grounds evaluations in detailed comparisons.

Findings

The system achieves human-like novelty scoring.

It reduces overestimation of novelty.

It improves consistency over existing methods.

Abstract

Assessing research novelty is a core yet highly subjective aspect of peer review, typically based on implicit judgment and incomplete comparison to prior work. We introduce a literature-aware novelty assessment framework that explicitly learns how humans judge novelty from peer-review reports and grounds these judgments in structured comparison to existing research. Using nearly 80K novelty-annotated reviews from top-tier AI conferences, we fine-tune a large language model to capture reviewer-aligned novelty evaluation behavior. For a given manuscript, the system extracts structured representations of its ideas, methods, and claims, retrieves semantically related papers, and constructs a similarity graph that enables fine-grained, concept-level comparison to prior work. Conditioning on this structured evidence, the model produces calibrated novelty scores and human-like explanatory assessments, reducing overestimation and improving consistency relative to existing approaches.