EduVQA: Towards Concept-Aware Assessment of Educational AI-Generated Videos

TL;DR

EduVQA introduces a concept-aware framework and benchmark for assessing educational AI-generated videos, focusing on subtle concept correctness alongside perceptual quality.

Contribution

It presents the first benchmark for concept-aware educational video assessment and a novel S2D-MoE model that captures fine-grained concept inconsistencies.

Findings

EduVQA outperforms existing methods in perceptual and semantic evaluation.

The benchmark contains 1,130 videos with over 310,650 annotations.

The code and dataset are publicly available at https://github.com/EduVQA/EduVQA.

Abstract

Existing AI-generated video quality assessment (AIGVQA) methods mainly focus on global perceptual realism and coarse text-video alignment, while overlooking a critical requirement in educational scenarios: concept correctness. In early mathematics education, subtle errors in numerical quantities, geometric relations, or spatial configurations may fundamentally alter the conveyed knowledge despite visually plausible generation. To address this problem, we introduce EduAVQABench, the first benchmark for concept-aware educational AIGV assessment, containing 1,130 videos generated by ten state-of-the-art T2V models together with over 310,650 fine-grained human annotations spanning perceptual quality and semantic alignment. Built upon this benchmark, we further propose EduVQA, a concept-aware AIGVQA framework equipped with a Structured 2D Mixture-of-Experts (S2D-MoE) architecture. By jointly modeling fine-grained concept assessment and overall quality prediction through shared experts and adaptive two-dimensional routing, EduVQA effectively captures subtle concept-level inconsistencies overlooked by conventional global scoring methods. Extensive experiments demonstrate that EduVQA consistently outperforms existing AIGVQA approaches across both perceptual and semantic evaluation tasks while exhibiting strong generalization capability on unseen benchmarks. Code and dataset will be publicly available at: https://github.com/EduVQA/EduVQA.