Embedded Graph Convolutional Networks for Real-Time Event Data Processing on SoC FPGAs

TL;DR

This paper presents EFGCN, a hardware-optimized graph convolutional network for event data processing on SoC FPGAs, achieving high throughput, low latency, and significant model size reduction for real-time embedded applications.

Contribution

The paper introduces a novel FPGA-accelerated GCN tailored for event cameras, with hardware-aware optimizations that drastically reduce model size and enable real-time processing on embedded platforms.

Findings

Up to 100-fold model size reduction compared to previous GNNs.

Achieved 13.3 million events per second throughput on FPGA.

Maintained low accuracy loss (around 2-3%) on classification tasks.

Abstract

The utilisation of event cameras represents an important and swiftly evolving trend aimed at addressing the constraints of traditional video systems. Particularly within the automotive domain, these cameras find significant relevance for their integration into embedded real-time systems due to lower latency and energy consumption. One effective approach to ensure the necessary throughput and latency for event processing is through the utilisation of graph convolutional networks (GCNs). In this study, we introduce a custom EFGCN (Event-based FPGA-accelerated Graph Convolutional Network) designed with a series of hardware-aware optimisations tailored for PointNetConv, a graph convolution designed for point cloud processing. The proposed techniques result in up to 100-fold reduction in model size compared to Asynchronous Event-based GNN (AEGNN), one of the most recent works in the field, with a relatively small decrease in accuracy (2.9% for the N-Caltech101 classification task, 2.2% for the N-Cars classification task), thus following the TinyML trend. We implemented EFGCN on a ZCU104 SoC FPGA platform without any external memory resources, achieving a throughput of 13.3 million events per second (MEPS) and real-time partially asynchronous processing with low latency. Our approach achieves state-of-the-art performance across multiple event-based classification benchmarks while remaining highly scalable, customisable and resource-efficient. We publish both software and hardware source code in an open repository: https://github.com/vision-agh/gcnn-dvs-fpga