Towards an Intelligent Assistive System Based on Augmented Reality and Serious Games

TL;DR

This paper presents a preliminary intelligent assistive system for the elderly with cognitive impairments, integrating IoT, augmented reality, and fuzzy decision-making to support daily activities and assess cognitive state.

Contribution

It introduces a multi-mode assistive system combining AR, IoT, and adaptive decision-making, with preliminary validation on elderly users and comparison to traditional cognitive tests.

Findings

Serious game scores correlate acceptably with paper-based tests.

Semi-automated mode results in less data loss than automated mode.

System effectively supports daily activities and cognitive assessment.

Abstract

Age-related cognitive impairment is generally characterized by gradual memory loss and decision-making difficulties. The aim of this study is to investigate multi level support and suggest relevant helping means for the elderly with mild cognitive impairment as well as their caregivers as the primary end-users. This work reports preliminary results on an intelligent assistive system, achieved through the integration of Internet of Things, augmented reality, and adaptive fuzzy decision-making methods. The proposed system operates in different modes, including automated and semi-automated modes. The former helps the user complete their daily life activities by showing augmented reality messages or making automatic changes; while the latter allows manual changes after the real-time assessment of the user's cognitive state based on the augmented reality serious game score. We have also evaluated the accuracy of the serious game score with 37 elderly participants and compared it with users' paper-based cognitive test results. We further noted that there is an acceptable correlation between the paper-based test and users' serious game scores. Moreover, we observed that the system response in the semi-automated mode causes less data loss compared with the automated mode, as the number of active devices decreases.