Detecting Events of Daily Living Using Multimodal Data
Hyungik Oh, Ramesh Jain

TL;DR
This paper presents a novel multimodal data approach using semantic enrichment and knowledge graphs to automatically recognize and analyze daily living events from smartphone and wearable device data.
Contribution
It introduces an unobtrusive semantic context enrichment method with a knowledge graph for daily event recognition, including a new food recognition technique based on physical responses.
Findings
Successful event recognition with semantic enrichment.
Effective food classification from physical responses.
Demonstrated approach over 14 months of data from three users.
Abstract
Events are fundamental for understanding how people experience their lives. It is challenging, however, to automatically record all events in daily life. An understanding of multimedia signals allows recognizing events of daily living and getting their attributes as automatically as possible. In this paper, we consider the problem of recognizing a daily event by employing the commonly used multimedia data obtained from a smartphone and wearable device. We develop an unobtrusive approach to obtain latent semantic information from the data, and therefore an approach for daily event recognition based on semantic context enrichment. We represent the enrichment process through an event knowledge graph that semantically enriches a daily event from a low-level daily activity. To show a concrete example of this enrichment, we perform an experiment with eating activity, which may be one of the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsContext-Aware Activity Recognition Systems · Nutritional Studies and Diet · Time Series Analysis and Forecasting
