Natural Language based Context Modeling and Reasoning for Ubiquitous Computing with Large Language Models: A Tutorial

TL;DR

This tutorial explores how large language models can be used for context modeling and reasoning in ubiquitous computing, enabling applications like assisted living and personalized trip planning without model fine-tuning.

Contribution

It introduces the LLM-driven Context-aware Computing (LCaC) paradigm, demonstrating how prompts and autonomous agents enable context reasoning with LLMs without fine-tuning.

Findings

LLMs can effectively model and reason about context using natural language prompts.

AutoAgents facilitate context-aware decision making in real-world applications.

The approach is validated through showcases in assisted living and trip planning.

Abstract

Large language models (LLMs) have become phenomenally surging, since 2018--two decades after introducing context-awareness into computing systems. Through taking into account the situations of ubiquitous devices, users and the societies, context-aware computing has enabled a wide spectrum of innovative applications, such as assisted living, location-based social network services and so on. To recognize contexts and make decisions for actions accordingly, various artificial intelligence technologies, such as Ontology and OWL, have been adopted as representations for context modeling and reasoning. Recently, with the rise of LLMs and their improved natural language understanding and reasoning capabilities, it has become feasible to model contexts using natural language and perform context reasoning by interacting with LLMs such as ChatGPT and GPT-4. In this tutorial, we demonstrate the use of texts, prompts, and autonomous agents (AutoAgents) that enable LLMs to perform context modeling and reasoning without requiring fine-tuning of the model. We organize and introduce works in the related field, and name this computing paradigm as the LLM-driven Context-aware Computing (LCaC). In the LCaC paradigm, users' requests, sensors reading data, and the command to actuators are supposed to be represented as texts. Given the text of users' request and sensor data, the AutoAgent models the context by prompting and sends to the LLM for context reasoning. LLM generates a plan of actions and responds to the AutoAgent, which later follows the action plan to foster context-awareness. To prove the concepts, we use two showcases--(1) operating a mobile z-arm in an apartment for assisted living, and (2) planning a trip and scheduling the itinerary in a context-aware and personalized manner.