Overview of the SciHigh Track at FIRE 2025: Research Highlight Generation from Scientific Papers

TL;DR

This paper presents the SciHigh track at FIRE 2025, which benchmarks automatic generation of concise scientific paper highlights using models trained on the MixSub dataset, aiming to improve literature review efficiency.

Contribution

It introduces a new benchmark for scientific highlight generation, evaluates various models including pre-trained language models, and provides insights into their effectiveness.

Findings

Highlights can reduce reading effort

Generated highlights help in faster literature reviews

Models show promising results with ROUGE-L scores

Abstract

`SciHigh: Research Highlight Generation from Scientific Papers' focuses on the task of automatically generating concise, informative, and meaningful bullet-point highlights directly from scientific abstracts. The goal of this task is to evaluate how effectively computational models can generate highlights that capture the key contributions, findings, and novelty of a paper in a concise form. Highlights help readers grasp essential ideas quickly and are often easier to read and understand than longer paragraphs, especially on mobile devices. The track uses the MixSub dataset \cite{10172215}, which provides pairs of abstracts and corresponding author-written highlights. In this inaugural edition of the track, 12 teams participated, exploring various approaches, including pre-trained language models, to generate highlights from this scientific dataset. All submissions were evaluated using established metrics such as ROUGE, METEOR, and BERTScore to measure both alignment with author-written highlights and overall informativeness. Teams were ranked based on ROUGE-L scores. The findings suggest that automatically generated highlights can reduce reading effort, accelerate literature reviews, and enhance metadata for digital libraries and academic search platforms. SciHigh provides a dedicated benchmark for advancing methods aimed at concise and accurate highlight generation from scientific writing.