Video Summarisation with Incident and Context Information using Generative AI

TL;DR

This paper introduces a novel GenAI-based system for video summarization that combines object detection and comprehensive analysis to generate accurate, context-aware textual summaries for efficient video review.

Contribution

It presents a new approach integrating YOLO-V8 and Gemini with GenAI to produce tailored, context-rich video summaries, improving over conventional methods.

Findings

Achieved 72.8% similarity in summaries

Rated 85% accuracy in qualitative assessments

Enhanced contextual accuracy in video summaries

Abstract

The proliferation of video content production has led to vast amounts of data, posing substantial challenges in terms of analysis efficiency and resource utilization. Addressing this issue calls for the development of robust video analysis tools. This paper proposes a novel approach leveraging Generative Artificial Intelligence (GenAI) to facilitate streamlined video analysis. Our tool aims to deliver tailored textual summaries of user-defined queries, offering a focused insight amidst extensive video datasets. Unlike conventional frameworks that offer generic summaries or limited action recognition, our method harnesses the power of GenAI to distil relevant information, enhancing analysis precision and efficiency. Employing YOLO-V8 for object detection and Gemini for comprehensive video and text analysis, our solution achieves heightened contextual accuracy. By combining YOLO with Gemini, our approach furnishes textual summaries extracted from extensive CCTV footage, enabling users to swiftly navigate and verify pertinent events without the need for exhaustive manual review. The quantitative evaluation revealed a similarity of 72.8%, while the qualitative assessment rated an accuracy of 85%, demonstrating the capability of the proposed method.