LLM-assisted Agentic Edge Intelligence Framework

TL;DR

The paper introduces LEI, a framework that leverages cloud-based LLMs to generate and update device-specific logic for edge intelligence, enhancing adaptability and resource efficiency in heterogeneous environments.

Contribution

It presents a novel cloud-assisted system that dynamically creates and deploys lightweight, tailored programs to edge devices using LLMs, reducing manual coding and improving scalability.

Findings

LEI maintains low CPU and memory usage across diverse datasets.

The framework adapts efficiently to changing data and resource constraints.

Experimental results confirm LEI's resource-efficient and flexible operation.

Abstract

Edge intelligence delivers low-latency inference, yet most edge analytics remain hard-coded and must be redeployed as conditions change. When data patterns shift or new questions arise, engineers often need to write new scripts and push updates to devices, which slows iteration and raises operating costs. This limited adaptability reduces scalability and autonomy in large, heterogeneous, and resource-constrained edge deployments, and it increases reliance on human oversight. Meanwhile, large language models (LLMs) can interpret instructions and generate code, but their compute and memory requirements typically prevent direct deployment on edge devices. We address this gap with the LLM-assisted Edge Intelligence (LEI) framework, which removes the need for manually specified business logic. In LEI, a cloud-hosted LLM coordinates the creation and update of device-side logic as requirements evolve. The system generates candidate lightweight programs, checks them against available data and constraints, and then deploys the selected version to each device. This lets each device receive a tailored program based on sample data, metadata, context, and current resource limits. We evaluate LEI on four heterogeneous datasets, including air quality, temperature \& humidity, wind, and soil datasets using multiple LLM backends. The experimental results show that the framework maintains low average CPU and memory utilization during the execution. These results indicate that the framework adapts efficiently to changing conditions while maintaining resource efficiency.