High-Performance FPGA Implementation of Equivariant Adaptive Separation via Independence Algorithm for Independent Component Analysis

TL;DR

This paper introduces a novel FPGA implementation of an adaptive ICA algorithm that significantly enhances processing speed and throughput, enabling efficient real-time machine learning applications involving probability density functions.

Contribution

The paper presents a new algorithm and FPGA design that drastically improves the clock frequency and throughput of adaptive ICA, extending its applicability to broader machine learning tasks.

Findings

FPGA implementation increases clock frequency by over ten times.

Throughput is improved by at least a hundred times.

The algorithm is adaptable to various stochastic gradient descent-based machine learning problems.

Abstract

Independent Component Analysis (ICA) is a dimensionality reduction technique that can boost efficiency of machine learning models that deal with probability density functions, e.g. Bayesian neural networks. Algorithms that implement adaptive ICA converge slower than their nonadaptive counterparts, however, they are capable of tracking changes in underlying distributions of input features. This intrinsically slow convergence of adaptive methods combined with existing hardware implementations that operate at very low clock frequencies necessitate fundamental improvements in both algorithm and hardware design. This paper presents an algorithm that allows efficient hardware implementation of ICA. Compared to previous work, our FPGA implementation of adaptive ICA improves clock frequency by at least one order of magnitude and throughput by at least two orders of magnitude. Our proposed algorithm is not limited to ICA and can be used in various machine learning problems that use stochastic gradient descent optimization.