An Expert Schema for Evaluating Large Language Model Errors in Scholarly Question-Answering Systems

TL;DR

This paper introduces a structured schema for evaluating errors in large language models used in scholarly question-answering, aligning with expert assessment practices to improve reliability and detection of issues.

Contribution

The paper presents a validated error evaluation schema based on expert input, enhancing the assessment of LLM outputs in scholarly contexts.

Findings

Identified 20 error patterns across 7 categories through expert analysis.

Validated the schema with 10 scientists, showing improved error detection.

Schema supports personalized and context-aware evaluation tools.

Abstract

Large Language Models (LLMs) are transforming scholarly tasks like search and summarization, but their reliability remains uncertain. Current evaluation metrics for testing LLM reliability are primarily automated approaches that prioritize efficiency and scalability, but lack contextual nuance and fail to reflect how scientific domain experts assess LLM outputs in practice. We developed and validated a schema for evaluating LLM errors in scholarly question-answering systems that reflects the assessment strategies of practicing scientists. In collaboration with domain experts, we identified 20 error patterns across seven categories through thematic analysis of 68 question-answer pairs. We validated this schema through contextual inquiries with 10 additional scientists, which showed not only which errors experts naturally identify but also how structured evaluation schemas can help them detect previously overlooked issues. Domain experts use systematic assessment strategies, including technical precision testing, value-based evaluation, and meta-evaluation of their own practices. We discuss implications for supporting expert evaluation of LLM outputs, including opportunities for personalized, schema-driven tools that adapt to individual evaluation patterns and expertise levels.