FastLexRank: Efficient Lexical Ranking for Structuring Social Media Posts

TL;DR

FastLexRank is a scalable, efficient implementation of LexRank that reduces computational complexity from quadratic to linear, enabling real-time processing of large social media datasets without sacrificing accuracy.

Contribution

The paper introduces algorithmic improvements that significantly enhance LexRank's efficiency, making it suitable for large-scale, real-time social media text analysis.

Findings

Achieves linear time complexity for LexRank computations

Maintains identical ranking results as the original LexRank

Demonstrates effectiveness on social media datasets

Abstract

We present FastLexRank\footnote{https://github.com/LiMaoUM/FastLexRank}, an efficient and scalable implementation of the LexRank algorithm for text ranking. Designed to address the computational and memory complexities of the original LexRank method, FastLexRank significantly reduces time and memory requirements from $\mathcal{O}(n^2)$ to $\mathcal{O}(n)$ without compromising the quality or accuracy of the results. By employing an optimized approach to calculating the stationary distribution of sentence graphs, FastLexRank maintains an identical results with the original LexRank scores while enhancing computational efficiency. This paper details the algorithmic improvements that enable the processing of large datasets, such as social media corpora, in real-time. Empirical results demonstrate its effectiveness, and we propose its use in identifying central tweets, which can be further analyzed using advanced NLP techniques. FastLexRank offers a scalable solution for text centrality calculation, addressing the growing need for efficient processing of digital content.