OpenLifelogQA: An Open-Ended Multi-Modal Lifelog Question-Answering Dataset

TL;DR

OpenLifelogQA is a comprehensive, multimodal dataset of 14,187 question-answer pairs from 18 months of personal lifelog data, designed to advance open-ended question answering in lifelog applications.

Contribution

The paper introduces a large-scale, diverse lifelog QA dataset and provides baseline evaluations with state-of-the-art models for realistic personal data querying.

Findings

Achieved 89.7% BERTScore on baseline evaluation.

Demonstrated the dataset's diversity and practicality for real-world applications.

Provided benchmarks for future lifelog QA research.

Abstract

We introduce OpenLifelogQA, a large-scale open-ended lifelog QA dataset constructed from 18 months of multimodal lifelog data. Lifelogging is the passive collection and analysis of personal daily activities using wearable devices, producing rich multimodal data such as images, locations, and biometrics. Question answering (QA) over lifelog data enables users to interactively query their own experiences, supporting applications in memory support, lifestyle analysis, and personal assistance. OpenLifelogQA contains 14,187 Q&A pairs spanning multiple question types and difficulty levels, designed to support robust evaluation in realistic settings. Compared with prior resources, OpenLifelogQA offers greater diversity and practicality for real-world applications. To establish baselines, we evaluate the LLaVA-NeXT-Interleave 7B model, achieving 89.7% BERTScore, 25.87% ROUGE-L, and an average LLM Score of 3.97. By releasing OpenLifelogQA, we aim to promote future research on lifelog technologies, paving the way for personal lifelog assistants capable of memory augmentation, healthcare support, and lifestyle coaching.