Configurable p-Neurons Using Modular p-Bits
Saleh Bunaiyan, Mohammad Alsharif, Abdelrahman S. Abdelrahman, Hesham ElSawy, Suraj S. Cheema, Suhaib A. Fahmy, Kerem Y. Camsari, Feras Al-Dirini

TL;DR
This paper introduces a modular p-bit design that enables configurable probabilistic activation functions for neural networks, demonstrated through spintronic and FPGA implementations, offering significant hardware resource savings.
Contribution
The paper re-engineers p-bits into modular units allowing a range of probabilistic activation functions, including sigmoid, Tanh, and ReLU, with practical spintronic and FPGA implementations.
Findings
Wide and tunable probabilistic ranges achieved.
Order of magnitude hardware resource savings demonstrated.
Successful FPGA implementation with shared stochastic units.
Abstract
Probabilistic bits (p-bits) have recently been employed in neural networks (NNs) as stochastic neurons with sigmoidal probabilistic activation functions. Nonetheless, there remain a wealth of other probabilistic activation functions that are yet to be explored. Here we re-engineer the p-bit by decoupling its stochastic signal path from its input data path, giving rise to a modular p-bit that enables the realization of probabilistic neurons (p-neurons) with a range of configurable probabilistic activation functions, including a probabilistic version of the widely used Logistic Sigmoid, Tanh and Rectified Linear Unit (ReLU) activation functions. We present spintronic (CMOS + sMTJ) designs that show wide and tunable probabilistic ranges of operation. Finally, we experimentally implement digital-CMOS versions on an FPGA, with stochastic unit sharing, and demonstrate an order of magnitude…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Numerical Methods and Algorithms · Advanced Memory and Neural Computing
