TR-EduVSum: A Turkish-Focused Dataset and Consensus Framework for Educational Video Summarization

TL;DR

This paper introduces TR-EduVSum, a new Turkish educational video dataset and a consensus framework AutoMUP for automatic, reproducible video summarization based on multiple human summaries.

Contribution

It presents a novel dataset and a pyramid-inspired consensus method AutoMUP that generates high-quality summaries from multiple human annotations.

Findings

AutoMUP summaries have high semantic overlap with LLM summaries.

Consensus weight and clustering are crucial for summary quality.

The approach can be adapted to other Turkic languages.

Abstract

This study presents a framework for generating the gold-standard summary fully automatically and reproducibly based on multiple human summaries of Turkish educational videos. Within the scope of the study, a new dataset called TR-EduVSum was created, encompassing 82 Turkish course videos in the field of "Data Structures and Algorithms" and containing a total of 3281 independent human summaries. Inspired by existing pyramid-based evaluation approaches, the AutoMUP (Automatic Meaning Unit Pyramid) method is proposed, which extracts consensus-based content from multiple human summaries. AutoMUP clusters the meaning units extracted from human summaries using embedding, statistically models inter-participant agreement, and generates graded summaries based on consensus weight. In this framework, the gold summary corresponds to the highest-consensus AutoMUP configuration, constructed from the most frequently supported meaning units across human summaries. Experimental results show that AutoMUP summaries exhibit high semantic overlap with robust LLM (Large Language Model) summaries such as Flash 2.5 and GPT-5.1. Furthermore, ablation studies clearly demonstrate the decisive role of consensus weight and clustering in determining summary quality. The proposed approach can be generalized to other Turkic languages at low cost.