A Hardware-Efficient ADMM-Based SVM Training Algorithm for Edge Computing

TL;DR

This paper presents a hardware-efficient ADMM-based SVM training algorithm that leverages low-rank approximation and optimized hardware design, significantly reducing training latency, memory, and energy consumption for edge computing applications.

Contribution

It introduces a novel ADMM-based SVM training method with low-rank approximation and hardware optimization, enabling low-power, real-time training on edge devices.

Findings

Reduces matrix dimension by 32× with minimal accuracy loss

Achieves 9.8×10^7 times shorter training latency compared to SMO

Provides 153,310× higher energy efficiency in a dedicated chip

Abstract

This work demonstrates a hardware-efficient support vector machine (SVM) training algorithm via the alternative direction method of multipliers (ADMM) optimizer. Low-rank approximation is exploited to reduce the dimension of the kernel matrix by employing the Nystr\"{o}m method. Verified in four datasets, the proposed ADMM-based training algorithm with rank approximation reduces 32$\times$ of matrix dimension with only 2% drop in inference accuracy. Compared to the conventional sequential minimal optimization (SMO) algorithm, the ADMM-based training algorithm is able to achieve a 9.8$\times$10$^7$ shorter latency for training 2048 samples. Hardware design techniques, including pre-computation and memory sharing, are proposed to reduce the computational complexity by 62% and the memory usage by 60%. As a proof of concept, an epileptic seizure detector chip is designed to demonstrate the effectiveness of the proposed hardware-efficient training algorithm. The chip achieves a 153,310$\times$ higher energy efficiency and a 364$\times$ higher throughput-to-area ratio for SVM training than a high-end CPU. This work provides a promising solution for edge devices which require low-power and real-time training.