Computing High-Degree Polynomial Gradients in Memory

TL;DR

This paper introduces a scalable, memory-efficient hardware approach for computing gradients of high-degree polynomials, enabling faster and more energy-efficient optimization in machine learning and control applications.

Contribution

It proposes a novel parallel gradient computation method for high-degree polynomials suitable for in-memory hardware implementation, scalable with problem size and degree.

Findings

Demonstrated solving a 3rd-order Boolean satisfiability problem with memristor crossbar circuits.

Simulation shows significant improvements in area, speed, and energy efficiency over existing methods.

Approach is applicable to larger, practical problems in optimization and machine learning.

Abstract

Specialized function gradient computing hardware could greatly improve the performance of state-of-the-art optimization algorithms, e.g., based on gradient descent or conjugate gradient methods that are at the core of control, machine learning, and operations research applications. Prior work on such hardware, performed in the context of the Ising Machines and related concepts, is limited to quadratic polynomials and not scalable to commonly used higher-order functions. Here, we propose a novel approach for massively parallel gradient calculations of high-degree polynomials, which is conducive to efficient mixed-signal in-memory computing circuit implementations and whose area complexity scales linearly with the number of variables and terms in the function and, most importantly, independent of its degree. Two flavors of such an approach are proposed. The first is limited to binary-variable polynomials typical in combinatorial optimization problems, while the second type is broader at the cost of a more complex periphery. To validate the former approach, we experimentally demonstrated solving a small-scale third-order Boolean satisfiability problem based on integrated metal-oxide memristor crossbar circuits, one of the most prospective in-memory computing device technologies, with a competitive heuristics algorithm. Simulation results for larger-scale, more practical problems show orders of magnitude improvements in the area, and related advantages in speed and energy efficiency compared to the state-of-the-art. We discuss how our work could enable even higher-performance systems after co-designing algorithms to exploit massively parallel gradient computation.