Large Language Model Based Multi-Agent System Augmented Complex Event Processing Pipeline for Internet of Multimedia Things

TL;DR

This paper develops a multi-agent system leveraging large language models for complex event processing in multimedia IoT, demonstrating a scalable, autonomous pipeline with insights into performance trade-offs.

Contribution

It introduces a novel LLM-based multi-agent CEP framework integrated with pub/sub tools, advancing the automation and scalability of multimedia IoT event processing.

Findings

Higher agent count increases latency

System maintains narrative coherence at scale

Performance varies with video complexity

Abstract

This paper presents the development and evaluation of a Large Language Model (LLM), also known as foundation models, based multi-agent system framework for complex event processing (CEP) with a focus on video query processing use cases. The primary goal is to create a proof-of-concept (POC) that integrates state-of-the-art LLM orchestration frameworks with publish/subscribe (pub/sub) tools to address the integration of LLMs with current CEP systems. Utilizing the Autogen framework in conjunction with Kafka message brokers, the system demonstrates an autonomous CEP pipeline capable of handling complex workflows. Extensive experiments evaluate the system's performance across varying configurations, complexities, and video resolutions, revealing the trade-offs between functionality and latency. The results show that while higher agent count and video complexities increase latency, the system maintains high consistency in narrative coherence. This research builds upon and contributes to, existing novel approaches to distributed AI systems, offering detailed insights into integrating such systems into existing infrastructures.