SECite: Analyzing and Summarizing Citations in Software Engineering Literature

TL;DR

SECite introduces a novel NLP-based framework that analyzes citation sentiment and generates summaries to evaluate the impact and perception of research papers in software engineering literature.

Contribution

The paper presents SECite, a semi-automated pipeline combining sentiment analysis and generative AI to assess scholarly impact through citation contexts.

Findings

Reveals patterns in academic perception of research papers

Highlights divergence between citation sentiment and authors' presentation

Provides a comprehensive framework for impact assessment

Abstract

Identifying the strengths and limitations of a research paper is a core component of any literature review. However, traditional summaries reflect only the authors' self-presented perspective. Analyzing how other researchers discuss and cite the paper can offer a deeper, more practical understanding of its contributions and shortcomings. In this research, we introduce SECite, a novel approach for evaluating scholarly impact through sentiment analysis of citation contexts. We develop a semi-automated pipeline to extract citations referencing nine research papers and apply advanced natural language processing (NLP) techniques with unsupervised machine learning to classify these citation statements as positive or negative. Beyond sentiment classification, we use generative AI to produce sentiment-specific summaries that capture the strengths and limitations of each target paper, derived both from clustered citation groups and from the full text. Our findings reveal meaningful patterns in how the academic community perceives these works, highlighting areas of alignment and divergence between external citation feedback and the authors' own presentation. By integrating citation sentiment analysis with LLM-based summarization, this study provides a comprehensive framework for assessing scholarly contributions.