Edge AI Collaborative Learning: Bayesian Approaches to Uncertainty Estimation

TL;DR

This paper explores Bayesian neural networks in distributed edge AI systems, demonstrating their effectiveness for uncertainty estimation in collaborative mapping tasks within simulated environments.

Contribution

It introduces a novel integration of Bayesian neural networks with distributed learning algorithms like DiNNO for uncertainty estimation in edge AI applications.

Findings

BNNs effectively support uncertainty estimation in distributed learning.

Applying KL divergence reduces validation loss by 12-30%.

Hyperparameter tuning is critical for optimal uncertainty assessment.

Abstract

Recent advancements in edge computing have significantly enhanced the AI capabilities of Internet of Things (IoT) devices. However, these advancements introduce new challenges in knowledge exchange and resource management, particularly addressing the spatiotemporal data locality in edge computing environments. This study examines algorithms and methods for deploying distributed machine learning within autonomous, network-capable, AI-enabled edge devices. We focus on determining confidence levels in learning outcomes considering the spatial variability of data encountered by independent agents. Using collaborative mapping as a case study, we explore the application of the Distributed Neural Network Optimization (DiNNO) algorithm extended with Bayesian neural networks (BNNs) for uncertainty estimation. We implement a 3D environment simulation using the Webots platform to simulate collaborative mapping tasks, decouple the DiNNO algorithm into independent processes for asynchronous network communication in distributed learning, and integrate distributed uncertainty estimation using BNNs. Our experiments demonstrate that BNNs can effectively support uncertainty estimation in a distributed learning context, with precise tuning of learning hyperparameters crucial for effective uncertainty assessment. Notably, applying Kullback-Leibler divergence for parameter regularization resulted in a 12-30% reduction in validation loss during distributed BNN training compared to other regularization strategies.