SKT5SciSumm -- Revisiting Extractive-Generative Approach for Multi-Document Scientific Summarization

TL;DR

This paper introduces SKT5SciSumm, a hybrid multi-document scientific summarization framework that combines sentence embeddings and T5 models, achieving state-of-the-art results efficiently.

Contribution

The paper presents a novel hybrid approach using citation-informed embeddings and T5 models for effective extractive and abstractive scientific summarization.

Findings

Achieves state-of-the-art performance on Multi-XScience dataset

Uses less complex models to obtain remarkable results

Demonstrates the effectiveness of combining embeddings with T5 for scientific summarization

Abstract

Summarization for scientific text has shown significant benefits both for the research community and human society. Given the fact that the nature of scientific text is distinctive and the input of the multi-document summarization task is substantially long, the task requires sufficient embedding generation and text truncation without losing important information. To tackle these issues, in this paper, we propose SKT5SciSumm - a hybrid framework for multi-document scientific summarization (MDSS). We leverage the Sentence-Transformer version of Scientific Paper Embeddings using Citation-Informed Transformers (SPECTER) to encode and represent textual sentences, allowing for efficient extractive summarization using k-means clustering. We employ the T5 family of models to generate abstractive summaries using extracted sentences. SKT5SciSumm achieves state-of-the-art performance on the Multi-XScience dataset. Through extensive experiments and evaluation, we showcase the benefits of our model by using less complicated models to achieve remarkable results, thereby highlighting its potential in advancing the field of multi-document summarization for scientific text.