Leveraging LLMs to Predict Affective States via Smartphone Sensor Features

TL;DR

This paper explores the novel use of large language models to predict mental health-related affective states from smartphone sensor data, demonstrating promising results in digital phenotyping for early mental health detection.

Contribution

It introduces the first application of LLMs for affective state prediction using smartphone data, highlighting their potential in digital mental health monitoring.

Findings

LLMs can predict affective states from smartphone data.

Zero-shot and few-shot LLM embeddings are effective.

This approach offers a new avenue for digital phenotyping.

Abstract

As mental health issues for young adults present a pressing public health concern, daily digital mood monitoring for early detection has become an important prospect. An active research area, digital phenotyping, involves collecting and analysing data from personal digital devices such as smartphones (usage and sensors) and wearables to infer behaviours and mental health. Whilst this data is standardly analysed using statistical and machine learning approaches, the emergence of large language models (LLMs) offers a new approach to make sense of smartphone sensing data. Despite their effectiveness across various domains, LLMs remain relatively unexplored in digital mental health, particularly in integrating mobile sensor data. Our study aims to bridge this gap by employing LLMs to predict affect outcomes based on smartphone sensing data from university students. We demonstrate the efficacy of zero-shot and few-shot embedding LLMs in inferring general wellbeing. Our findings reveal that LLMs can make promising predictions of affect measures using solely smartphone sensing data. This research sheds light on the potential of LLMs for affective state prediction, emphasizing the intricate link between smartphone behavioral patterns and affective states. To our knowledge, this is the first work to leverage LLMs for affective state prediction and digital phenotyping tasks.