Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training

TL;DR

This paper introduces a dynamic symmetric point tracking method for analog in-memory computing, addressing calibration challenges and improving training accuracy in energy-efficient AI hardware.

Contribution

It provides the first theoretical analysis of pulse complexity for SP calibration and proposes a novel dynamic tracking approach with convergence guarantees.

Findings

The method reduces calibration pulse complexity.

Dynamic tracking improves training accuracy.

Enhanced variant with chopping and filtering enhances robustness.

Abstract

Analog in-memory computing (AIMC) performs computation directly within resistive crossbar arrays, offering an energy-efficient platform to scale large vision and language models. However, non-ideal analog device properties make the training on AIMC devices challenging. In particular, its update asymmetry can induce a systematic drift of weight updates towards a device-specific symmetric point (SP), which typically does not align with the optimum of the training objective. To mitigate this bias, most existing works assume the SP is known and pre-calibrate it to zero before training by setting the reference point as the SP. Nevertheless, calibrating AIMC devices requires costly pulse updates, and residual calibration error can directly degrade training accuracy. In this work, we present the first theoretical characterization of the pulse complexity of SP calibration and the resulting estimation error. We further propose a dynamic SP estimation method that tracks the SP during model training, and establishes its convergence guarantees. In addition, we develop an enhanced variant based on chopping and filtering techniques from digital signal processing. Numerical experiments demonstrate both the efficiency and effectiveness of the proposed method.