Hierarchical Hidden Markov Model in Detecting Activities of Daily Living in Wearable Videos for Studies of Dementia

TL;DR

This paper introduces a hierarchical Hidden Markov Model approach to automatically segment and structure long wearable videos of daily activities, aiding dementia diagnosis by enabling efficient navigation through challenging video data.

Contribution

The work presents a novel hierarchical HMM method for activity detection in complex wearable videos, integrating motion, visual, and audio features for improved structuring.

Findings

Effective segmentation of long, challenging videos

Promising activity detection results on real patient data

Demonstrates potential for aiding medical diagnosis

Abstract

This paper presents a method for indexing activities of daily living in videos obtained from wearable cameras. In the context of dementia diagnosis by doctors, the videos are recorded at patients' houses and later visualized by the medical practitioners. The videos may last up to two hours, therefore a tool for an efficient navigation in terms of activities of interest is crucial for the doctors. The specific recording mode provides video data which are really difficult, being a single sequence shot where strong motion and sharp lighting changes often appear. Our work introduces an automatic motion based segmentation of the video and a video structuring approach in terms of activities by a hierarchical two-level Hidden Markov Model. We define our description space over motion and visual characteristics of video and audio channels. Experiments on real data obtained from the recording at home of several patients show the difficulty of the task and the promising results of our approach.