EF-Train: Enable Efficient On-device CNN Training on FPGA Through Data Reshaping for Online Adaptation or Personalization

TL;DR

EF-Train is a novel FPGA-based accelerator enabling efficient on-device CNN training for real-time adaptation and personalization on resource-limited edge devices, using data reshaping and unified parallelism.

Contribution

The paper introduces EF-Train, a unified channel-level parallelism-based convolution kernel and data reshaping technique for efficient on-device CNN training on low-power FPGAs.

Findings

Achieves 46.99 GFLOPS throughput

Attains 6.09 GFLOPS/W energy efficiency

Supports end-to-end training on resource-limited FPGAs

Abstract

Conventionally, DNN models are trained once in the cloud and deployed in edge devices such as cars, robots, or unmanned aerial vehicles (UAVs) for real-time inference. However, there are many cases that require the models to adapt to new environments, domains, or new users. In order to realize such domain adaption or personalization, the models on devices need to be continuously trained on the device. In this work, we design EF-Train, an efficient DNN training accelerator with a unified channel-level parallelism-based convolution kernel that can achieve end-to-end training on resource-limited low-power edge-level FPGAs. It is challenging to implement on-device training on resource-limited FPGAs due to the low efficiency caused by different memory access patterns among forward, backward propagation, and weight update. Therefore, we developed a data reshaping approach with intra-tile continuous memory allocation and weight reuse. An analytical model is established to automatically schedule computation and memory resources to achieve high energy efficiency on edge FPGAs. The experimental results show that our design achieves 46.99 GFLOPS and 6.09GFLOPS/W in terms of throughput and energy efficiency, respectively.