Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis

TL;DR

This paper introduces over-the-air PCA that leverages channel noise to accelerate convergence in distributed principal component analysis, using a novel power-control scheme to adapt noise levels and improve learning speed.

Contribution

It proposes a new over-the-air PCA method that exploits channel noise for faster convergence, with a power-control scheme to adapt noise levels based on descent regions.

Findings

Channel noise can be exploited to accelerate gradient descent.

The proposed power-control scheme improves convergence speed.

Over-the-air PCA reduces multi-access latency.

Abstract

Recently years, the attempts on distilling mobile data into useful knowledge has been led to the deployment of machine learning algorithms at the network edge. Principal component analysis (PCA) is a classic technique for extracting the linear structure of a dataset, which is useful for feature extraction and data compression. In this work, we propose the deployment of distributed PCA over a multi-access channel based on the algorithm of stochastic gradient descent to learn the dominant feature space of a distributed dataset at multiple devices. Over-the-air aggregation is adopted to reduce the multi-access latency, giving the name over-the-air PCA. The novelty of this design lies in exploiting channel noise to accelerate the descent in the region around each saddle point encountered by gradient descent, thereby increasing the convergence speed of over-the-air PCA. The idea is materialized by proposing a power-control scheme which detects the type of descent region and controlling the level of channel noise accordingly. The scheme is proved to achieve a faster convergence rate than in the case without power control.