Efficient Video Summarization Framework using EEG and Eye-tracking Signals

TL;DR

This paper introduces a human-centered video summarization method that leverages EEG and eye-tracking data to identify key frames, achieving high accuracy and efficiency by incorporating human attention insights.

Contribution

The study presents a novel approach combining EEG and eye-tracking signals to improve video summarization, emphasizing human visual attention over traditional computer vision methods.

Findings

Achieves 96.5% video summarization accuracy

Outperforms state-of-the-art techniques in precision and recall

Reduces computational cost significantly

Abstract

This paper proposes an efficient video summarization framework that will give a gist of the entire video in a few key-frames or video skims. Existing video summarization frameworks are based on algorithms that utilize computer vision low-level feature extraction or high-level domain level extraction. However, being the ultimate user of the summarized video, humans remain the most neglected aspect. Therefore, the proposed paper considers human's role in summarization and introduces human visual attention-based summarization techniques. To understand human attention behavior, we have designed and performed experiments with human participants using electroencephalogram (EEG) and eye-tracking technology. The EEG and eye-tracking data obtained from the experimentation are processed simultaneously and used to segment frames containing useful information from a considerable video volume. Thus, the frame segmentation primarily relies on the cognitive judgments of human beings. Using our approach, a video is summarized by 96.5% while maintaining higher precision and high recall factors. The comparison with the state-of-the-art techniques demonstrates that the proposed approach yields ceiling-level performance with reduced computational cost in summarising the videos.