Leveraging BERT for Extractive Text Summarization on Lectures

TL;DR

This paper presents a Python-based RESTful service that uses BERT embeddings and KMeans clustering to generate extractive summaries of lecture content, aiming to improve summarization quality and usability for students.

Contribution

It introduces a novel lecture summarization service leveraging BERT and clustering, enhancing extractive summarization with modern deep learning techniques.

Findings

BERT-based embeddings improved summary relevance.

The service allows user-defined summary length.

Some limitations in model performance suggest future research directions.

Abstract

In the last two decades, automatic extractive text summarization on lectures has demonstrated to be a useful tool for collecting key phrases and sentences that best represent the content. However, many current approaches utilize dated approaches, producing sub-par outputs or requiring several hours of manual tuning to produce meaningful results. Recently, new machine learning architectures have provided mechanisms for extractive summarization through the clustering of output embeddings from deep learning models. This paper reports on the project called Lecture Summarization Service, a python based RESTful service that utilizes the BERT model for text embeddings and KMeans clustering to identify sentences closes to the centroid for summary selection. The purpose of the service was to provide students a utility that could summarize lecture content, based on their desired number of sentences. On top of the summary work, the service also includes lecture and summary management, storing content on the cloud which can be used for collaboration. While the results of utilizing BERT for extractive summarization were promising, there were still areas where the model struggled, providing feature research opportunities for further improvement.