An AI-Based System Utilizing IoT-Enabled Ambient Sensors and LLMs for Complex Activity Tracking

TL;DR

This paper introduces a non-invasive ambient sensing system that combines IoT sensors and large language models to recognize and reason about complex activities in elderly care, enhancing assistance and interaction.

Contribution

It presents a novel integration of ambient sensors with LLMs on edge devices for complex activity recognition and reasoning in elderly care.

Findings

Effective detection of multiple activities

Enhanced reasoning about activity sequences

Improved elderly assistance capabilities

Abstract

Complex activity recognition plays an important role in elderly care assistance. However, the reasoning ability of edge devices is constrained by the classic machine learning model capacity. In this paper, we present a non-invasive ambient sensing system that can detect multiple activities and apply large language models (LLMs) to reason the activity sequences. This method effectively combines edge devices and LLMs to help elderly people in their daily activities, such as reminding them to take pills or handling emergencies like falls. The LLM-based edge device can also serve as an interface to interact with elderly people, especially with memory issue, assisting them in their daily lives. By deploying such a system, we believe that the smart sensing system can improve the quality of life for older people and provide more efficient protection