MEVA: A Large-Scale Multiview, Multimodal Video Dataset for Activity Detection

TL;DR

The MEVA dataset is a comprehensive, large-scale multiview, multimodal video collection designed for advancing human activity recognition, featuring diverse scenarios, multiple modalities, and extensive annotations for research and benchmarking.

Contribution

We introduce MEVA, a large-scale, multimodal, multiview dataset with extensive annotations, enabling improved activity detection research and benchmarking in complex real-world scenarios.

Findings

Over 9300 hours of continuous video data collected.

Annotated 144 hours with 37 activity types and bounding boxes.

Includes diverse modalities: RGB, thermal IR, UAV footage, GPS.

Abstract

We present the Multiview Extended Video with Activities (MEVA) dataset, a new and very-large-scale dataset for human activity recognition. Existing security datasets either focus on activity counts by aggregating public video disseminated due to its content, which typically excludes same-scene background video, or they achieve persistence by observing public areas and thus cannot control for activity content. Our dataset is over 9300 hours of untrimmed, continuous video, scripted to include diverse, simultaneous activities, along with spontaneous background activity. We have annotated 144 hours for 37 activity types, marking bounding boxes of actors and props. Our collection observed approximately 100 actors performing scripted scenarios and spontaneous background activity over a three-week period at an access-controlled venue, collecting in multiple modalities with overlapping and non-overlapping indoor and outdoor viewpoints. The resulting data includes video from 38 RGB and thermal IR cameras, 42 hours of UAV footage, as well as GPS locations for the actors. 122 hours of annotation are sequestered in support of the NIST Activity in Extended Video (ActEV) challenge; the other 22 hours of annotation and the corresponding video are available on our website, along with an additional 306 hours of ground camera data, 4.6 hours of UAV data, and 9.6 hours of GPS logs. Additional derived data includes camera models geo-registering the outdoor cameras and a dense 3D point cloud model of the outdoor scene. The data was collected with IRB oversight and approval and released under a CC-BY-4.0 license.