ChainStream: An LLM-based Framework for Unified Synthetic Sensing

TL;DR

ChainStream leverages large language models to create a unified, transparent framework for personal context sensing, simplifying app development and improving data query accuracy through a feedback-guided optimizer.

Contribution

The paper introduces a novel LLM-based framework with a unified data processing approach and a feedback-guided query optimizer for context sensing tasks.

Findings

Efficient and precise automatic solving of 133 context sensing tasks.

The framework enhances transparency and ease of development for sensing applications.

Open-sourced code available for community use and benchmarking.

Abstract

Many applications demand context sensing to offer personalized and timely services. Yet, developing sensing programs can be challenging for developers and using them is privacy-concerning for end-users. In this paper, we propose to use natural language as the unified interface to process personal data and sense user context, which can effectively ease app development and make the data pipeline more transparent. Our work is inspired by large language models (LLMs) and other generative models, while directly applying them does not solve the problem - letting the model directly process the data cannot handle complex sensing requests and letting the model write the data processing program suffers error-prone code generation. We address the problem with 1) a unified data processing framework that makes context-sensing programs simpler and 2) a feedback-guided query optimizer that makes data query more informative. To evaluate the performance of natural language-based context sensing, we create a benchmark that contains 133 context sensing tasks. Extensive evaluation has shown that our approach is able to automatically solve the context-sensing tasks efficiently and precisely. The code is opensourced at https://github.com/MobileLLM/ChainStream.