Policies and Evaluation for Online Meeting Summarization

TL;DR

This paper introduces the first systematic study of online meeting summarization, proposing policies and metrics to evaluate real-time summaries, and demonstrating that adaptive policies outperform fixed ones on the AutoMin dataset.

Contribution

It pioneers the exploration of online meeting summarization, proposing policies, metrics, and demonstrating the effectiveness of adaptive strategies over fixed schedules.

Findings

Online models can produce strong summaries

Metrics enable detailed quality-latency analysis

Adaptive policies outperform fixed scheduled ones

Abstract

With more and more meetings moving to a digital domain, meeting summarization has recently gained interest in both academic and commercial research. However, prior academic research focuses on meeting summarization as an offline task, performed after the meeting concludes. In this paper, we perform the first systematic study of online meeting summarization. For this purpose, we propose several policies for conducting online summarization. We discuss the unique challenges of this task compared to the offline setting and define novel metrics to evaluate latency and partial summary quality. The experiments on the AutoMin dataset show that 1) online models can produce strong summaries, 2) our metrics allow a detailed analysis of different systems' quality-latency trade-off, also taking into account intermediate outputs and 3) adaptive policies perform better than fixed scheduled ones. These findings provide a starting point for the wider research community to explore this important task.