A Domain-Specific Language for LLM-Driven Trigger Generation in Multimodal Data Collection

TL;DR

This paper introduces a declarative, intent-driven framework using a domain-specific language and large language models to selectively and efficiently collect multimodal sensor data in real-time systems.

Contribution

It presents a novel DSL-based approach combined with LLMs for verifiable, high-level control of multimodal data collection on resource-constrained devices.

Findings

Higher generation consistency compared to unconstrained code generation

Lower execution latency while maintaining detection performance

Supports modular trigger composition and deployment on edge platforms

Abstract

Data-driven systems depend on task-relevant data, yet data collection pipelines remain passive and indiscriminate. Continuous logging of multimodal sensor streams incurs high storage costs and captures irrelevant data. This paper proposes a declarative framework for intent-driven, on-device data collection that enables selective collection of multimodal sensor data based on high-level user requests. The framework combines natural language interaction with a formally specified domain-specific language (DSL). Large language models translate user-defined requirements into verifiable and composable DSL programs that define conditional triggers across heterogeneous sensors, including cameras, LiDAR, and system telemetry. Empirical evaluation on vehicular and robotic perception tasks shows that the DSL-based approach achieves higher generation consistency and lower execution latency than unconstrained code generation while maintaining comparable detection performance. The structured abstraction supports modular trigger composition and concurrent deployment on resource-constrained edge platforms. This approach replaces passive logging with a verifiable, intent-driven mechanism for multimodal data collection in real-time systems.