State Space Models for Extractive Summarization in Low Resource Scenarios

TL;DR

This paper introduces MPoincareSum, a novel extractive summarization method utilizing state space models and Poincare compression to improve summary quality in low-resource scenarios, validated on Amazon reviews.

Contribution

The paper presents MPoincareSum, combining Mamba state space models with Poincare compression for effective extractive summarization in low-resource settings, outperforming existing methods.

Findings

MPoincareSum achieves higher ROUGE scores than baseline methods.

The approach effectively captures meaningful features in low-resource reviews.

Experimental results validate the superiority of the proposed method.

Abstract

Extractive summarization involves selecting the most relevant sentences from a text. Recently, researchers have focused on advancing methods to improve state-of-the-art results in low-resource settings. Motivated by these advancements, we propose the MPoincareSum method. This method applies the Mamba state space model to generate the semantics of reviews and sentences, which are then concatenated. A Poincare compression is used to select the most meaningful features, followed by the application of a linear layer to predict sentence relevance based on the corresponding review. Finally, we paraphrase the relevant sentences to create the final summary. To evaluate the effectiveness of MPoincareSum, we conducted extensive experiments using the Amazon review dataset. The performance of the method was assessed using ROUGE scores. The experimental results demonstrate that MPoincareSum outperforms several existing approaches in the literature