Cognitive Interpretation of Everyday Activities: Toward Perceptual Narrative Based Visuo-Spatial Scene Interpretation

TL;DR

This paper introduces a narrative-based computational model for high-level visuo-spatial scene understanding, focusing on human activity interpretation in smart environments to enhance assistive technologies and perceptual reasoning.

Contribution

It presents a novel narrative model for representing space, events, and actions, along with preliminary work on perceptual narrativisation for cognitive scene interpretation.

Findings

Developed a narrative model encompassing space, events, and actions.

Applied the model to human-activity interpretation in smart environments.

Outlined plans for open-source tools for perceptual narrativisation.

Abstract

We position a narrative-centred computational model for high-level knowledge representation and reasoning in the context of a range of assistive technologies concerned with "visuo-spatial perception and cognition" tasks. Our proposed narrative model encompasses aspects such as \emph{space, events, actions, change, and interaction} from the viewpoint of commonsense reasoning and learning in large-scale cognitive systems. The broad focus of this paper is on the domain of "human-activity interpretation" in smart environments, ambient intelligence etc. In the backdrop of a "smart meeting cinematography" domain, we position the proposed narrative model, preliminary work on perceptual narrativisation, and the immediate outlook on constructing general-purpose open-source tools for perceptual narrativisation. ACM Classification: I.2 Artificial Intelligence: I.2.0 General -- Cognitive Simulation, I.2.4 Knowledge Representation Formalisms and Methods, I.2.10 Vision and Scene Understanding: Architecture and control structures, Motion, Perceptual reasoning, Shape, Video analysis General keywords: cognitive systems; human-computer interaction; spatial cognition and computation; commonsense reasoning; spatial and temporal reasoning; assistive technologies