Audiovisual Database with 360 Video and Higher-Order Ambisonics Audio for Perception, Cognition, Behavior, and QoE Evaluation Research

TL;DR

This paper introduces a high-quality, open-source audiovisual database featuring 360-degree videos and Ambisonics audio, designed for research in perception, cognition, behavior, and QoE evaluation in immersive environments.

Contribution

It provides a publicly available, high-resolution audiovisual database with real-life scenes for systematic evaluation of multi-modal perception and experience in virtual environments.

Findings

Database includes 12 real-life scenes with 4th-order Ambisonics audio.

High-resolution 360 videos at 7680x3840, 60 fps, averaging 60 seconds per scene.

Database will be expanded with additional scenes, ratings, and saliency data.

Abstract

Research into multi-modal perception, human cognition, behavior, and attention can benefit from high-fidelity content that may recreate real-life-like scenes when rendered on head-mounted displays. Moreover, aspects of audiovisual perception, cognitive processes, and behavior may complement questionnaire-based Quality of Experience (QoE) evaluation of interactive virtual environments. Currently, there is a lack of high-quality open-source audiovisual databases that can be used to evaluate such aspects or systems capable of reproducing high-quality content. With this paper, we provide a publicly available audiovisual database consisting of twelve scenes capturing real-life nature and urban environments with a video resolution of 7680x3840 at 60 frames-per-second and with 4th-order Ambisonics audio. These 360 video sequences, with an average duration of 60 seconds, represent real-life settings for systematically evaluating various dimensions of uni-/multi-modal perception, cognition, behavior, and QoE. The paper provides details of the scene requirements, recording approach, and scene descriptions. The database provides high-quality reference material with a balanced focus on auditory and visual sensory information. The database will be continuously updated with additional scenes and further metadata such as human ratings and saliency information.