Toward Multimodal Interaction in Scalable Visual Digital Evidence Visualization Using Computer Vision Techniques and ISS

TL;DR

This paper proposes a scalable, multimodal visualization system for digital evidence in forensic investigations, integrating computer vision techniques with the Illimitable Space System to enhance evidence management and investigation workflows.

Contribution

It introduces a novel multimodal interaction framework using computer vision for scalable digital evidence visualization within the ISS platform, tailored for forensic case management.

Findings

Effective integration of gesture, eye-gaze, and audio interaction techniques.

Enhanced semantic linking and navigation of digital evidence.

Improved usability for forensic investigators.

Abstract

Visualization requirements in Forensic Lucid have to do with different levels of case knowledge abstraction, representation, aggregation, as well as the operational aspects as the final long-term goal of this proposal. It encompasses anything from the finer detailed representation of hierarchical contexts to Forensic Lucid programs, to the documented evidence and its management, its linkage to programs, to evaluation, and to the management of GIPSY software networks. This includes an ability to arbitrarily switch between those views combined with usable multimodal interaction. The purpose is to determine how the findings can be applied to Forensic Lucid and investigation case management. It is also natural to want a convenient and usable evidence visualization, its semantic linkage and the reasoning machinery for it. Thus, we propose a scalable management, visualization, and evaluation of digital evidence using the modified interactive 3D documentary system - Illimitable Space System - (ISS) to represent, semantically link, and provide a usable interface to digital investigators that is navigable via different multimodal interaction techniques using Computer Vision techniques including gestures, as well as eye-gaze and audio.