VideoAVE: A Multi-Attribute Video-to-Text Attribute Value Extraction Dataset and Benchmark Models

TL;DR

VideoAVE introduces a comprehensive video-to-text attribute value extraction dataset for e-commerce, along with benchmark models, highlighting the challenges and potential for future advancements in video-based attribute extraction.

Contribution

We present the first large-scale, multi-domain video AVE dataset with a filtering system and establish benchmark evaluations for state-of-the-art models.

Findings

VideoAVE covers 14 domains and 172 attributes.

Video AVE remains a challenging task, especially in open settings.

Current models have room for improvement in leveraging temporal information.

Abstract

Attribute Value Extraction (AVE) is important for structuring product information in e-commerce. However, existing AVE datasets are primarily limited to text-to-text or image-to-text settings, lacking support for product videos, diverse attribute coverage, and public availability. To address these gaps, we introduce VideoAVE, the first publicly available video-to-text e-commerce AVE dataset across 14 different domains and covering 172 unique attributes. To ensure data quality, we propose a post-hoc CLIP-based Mixture of Experts filtering system (CLIP-MoE) to remove the mismatched video-product pairs, resulting in a refined dataset of 224k training data and 25k evaluation data. In order to evaluate the usability of the dataset, we further establish a comprehensive benchmark by evaluating several state-of-the-art video vision language models (VLMs) under both attribute-conditioned value prediction and open attribute-value pair extraction tasks. Our results analysis reveals that video-to-text AVE remains a challenging problem, particularly in open settings, and there is still room for developing more advanced VLMs capable of leveraging effective temporal information. The dataset and benchmark code for VideoAVE are available at: https://github.com/gjiaying/VideoAVE