Distributed Parameter Estimation in Randomized One-hidden-layer Neural Networks

TL;DR

This paper proposes a fully distributed algorithm for parameter estimation in randomized one-hidden-layer neural networks, ensuring asymptotic unbiasedness and analyzing efficiency through variance bounds, validated on real-world energy data.

Contribution

It introduces a novel distributed estimation method with proven asymptotic unbiasedness and efficiency analysis for neural network parameters.

Findings

Estimator is asymptotically unbiased.

Variance of the estimation error has an established upper bound.

Empirical results on energy prediction data support theoretical claims.

Abstract

This paper addresses distributed parameter estimation in randomized one-hidden-layer neural networks. A group of agents sequentially receive measurements of an unknown parameter that is only partially observable to them. In this paper, we present a fully distributed estimation algorithm where agents exchange local estimates with their neighbors to collectively identify the true value of the parameter. We prove that this distributed update provides an asymptotically unbiased estimator of the unknown parameter, i.e., the first moment of the expected global error converges to zero asymptotically. We further analyze the efficiency of the proposed estimation scheme by establishing an asymptotic upper bound on the variance of the global error. Applying our method to a real-world dataset related to appliances energy prediction, we observe that our empirical findings verify the theoretical results.