How Can AI be Distributed in the Computing Continuum? Introducing the Neural Pub/Sub Paradigm

TL;DR

This paper introduces the neural pub/sub paradigm, a new approach to orchestrate AI workflows across large-scale distributed systems, addressing data management, resource allocation, and system resilience challenges in the computing continuum.

Contribution

It proposes a novel neural publish/subscribe framework to improve AI workflow management and system scalability in distributed environments, filling gaps left by traditional centralized methods.

Findings

Enhanced management of training, fine-tuning, and inference workflows

Improved distributed computation and resource allocation

Increased system resilience across the computing continuum

Abstract

This paper proposes the neural publish/subscribe paradigm, a novel approach to orchestrating AI workflows in large-scale distributed AI systems in the computing continuum. Traditional centralized broker methodologies are increasingly struggling with managing the data surge resulting from the proliferation of 5G systems, connected devices, and ultra-reliable applications. Moreover, the advent of AI-powered applications, particularly those leveraging advanced neural network architectures, necessitates a new approach to orchestrate and schedule AI processes within the computing continuum. In response, the neural pub/sub paradigm aims to overcome these limitations by efficiently managing training, fine-tuning and inference workflows, improving distributed computation, facilitating dynamic resource allocation, and enhancing system resilience across the computing continuum. We explore this new paradigm through various design patterns, use cases, and discuss open research questions for further exploration.