Fixed-Point Convolutional Neural Network for Real-Time Video Processing in FPGA

TL;DR

This paper introduces a fixed-point convolutional neural network architecture optimized for real-time video processing on FPGAs, emphasizing speed, hardware efficiency, and adaptability for practical camera-based image recognition tasks.

Contribution

The paper presents a novel neural network design with hardware-level optimizations and fixed-point arithmetic, enabling efficient real-time video processing on low-cost FPGAs.

Findings

Achieves real-time video processing on inexpensive FPGAs

Reduces network complexity with fixed-point arithmetic

Demonstrates effective adaptation to new datasets

Abstract

Modern mobile neural networks with a reduced number of weights and parameters do a good job with image classification tasks, but even they may be too complex to be implemented in an FPGA for video processing tasks. The article proposes neural network architecture for the practical task of recognizing images from a camera, which has several advantages in terms of speed. This is achieved by reducing the number of weights, moving from a floating-point to a fixed-point arithmetic, and due to a number of hardware-level optimizations associated with storing weights in blocks, a shift register, and an adjustable number of convolutional blocks that work in parallel. The article also proposed methods for adapting the existing data set for solving a different task. As the experiments showed, the proposed neural network copes well with real-time video processing even on the cheap FPGAs.