Perception Test 2024: Challenge Summary and a Novel Hour-Long VideoQA Benchmark
Joseph Heyward, Jo\~ao Carreira, Dima Damen, Andrew Zisserman, Viorica, P\u{a}tr\u{a}ucean

TL;DR
The paper reports on the second Perception Test challenge at ECCV 2024, benchmarking video understanding models across multiple modalities and tasks, and introduces a new hour-long video QA benchmark called 1h-walk VQA.
Contribution
It expands the Perception Test benchmark with seven tracks, including a novel hour-long video QA benchmark, to evaluate progress in video understanding models.
Findings
Benchmark results show progress in video model performance.
Introduction of the 1h-walk VQA benchmark for hour-long video understanding.
Diverse tasks across video, audio, and text modalities.
Abstract
Following the successful 2023 edition, we organised the Second Perception Test challenge as a half-day workshop alongside the IEEE/CVF European Conference on Computer Vision (ECCV) 2024, with the goal of benchmarking state-of-the-art video models and measuring the progress since last year using the Perception Test benchmark. This year, the challenge had seven tracks (up from six last year) and covered low-level and high-level tasks, with language and non-language interfaces, across video, audio, and text modalities; the additional track covered hour-long video understanding and introduced a novel video QA benchmark 1h-walk VQA. Overall, the tasks in the different tracks were: object tracking, point tracking, temporal action localisation, temporal sound localisation, multiple-choice video question-answering, grounded video question-answering, and hour-long video question-answering. We…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCCD and CMOS Imaging Sensors · Advanced Neural Network Applications · Industrial Vision Systems and Defect Detection
