Wearable AI in the Era of Large Sensor Models

TL;DR

This paper advocates for Large Sensor Models (LSMs) trained on multimodal wearable data as a unified, scalable framework to advance human-centric AI applications, addressing current modality silos and generalization challenges.

Contribution

It formalizes the concept of LSMs, analyzes their unique challenges, and proposes two research directions: with and without language capabilities, to foster future wearable AI development.

Findings

Identifies the potential of LSMs to unify multimodal wearable sensing

Highlights challenges in large-scale wearable data modeling

Suggests two pathways: LSMs with and without language capabilities

Abstract

As an effective approach to understanding the human-centric physical world, Wearable Artificial Intelligence (AI), which leverages multimodal wearable sensors to understand human physiology and behavior, has attracted increasing attention in recent years. However, existing sensor models remain largely siloed by modality and task, lacking a unified paradigm for integrating diverse wearable modalities, training strategies, and achieving robust generalization in real-world applications. Motivated by the success of multimodal foundation models, which learn transferable representations from massive multimodal data, we argue that Large Sensor Models (LSMs), defined as foundation models trained on large-scale and multimodal wearable data, offer a promising pathway toward a more general and scalable framework for wearable AI. In this position paper, we formalize the data substrate underlying LSMs, analyze the unique challenges of large-scale wearable sensing, and articulate two directions: (i) LSMs without language capability and (ii) LSMs with language capability. We further discuss representative application areas that can be unlocked by such models. Through this paper, we encourage the community to explore LSMs as a foundational approach for the next generation of human-centric AI systems.