Time-Shared Execution of Realtime Computer Vision Pipelines by Dynamic Partial Reconfiguration

TL;DR

This paper introduces a runtime framework for FPGA that enables multiple real-time computer vision pipelines to share the FPGA fabric dynamically through partial reconfiguration, maintaining real-time performance despite reconfiguration delays.

Contribution

It presents a novel FPGA runtime framework with optimizations for time-sharing multiple vision pipelines using dynamic partial reconfiguration, demonstrating feasibility and performance.

Findings

Achieves real-time performance with DPR on FPGA

Demonstrates effective time-sharing of FPGA for multiple pipelines

Provides a prototype on Xilinx ZC706 with performance evaluation

Abstract

This paper presents an FPGA runtime framework that demonstrates the feasibility of using dynamic partial reconfiguration (DPR) for time-sharing an FPGA by multiple realtime computer vision pipelines. The presented time-sharing runtime framework manages an FPGA fabric that can be round-robin time-shared by different pipelines at the time scale of individual frames. In this new use-case, the challenge is to achieve useful performance despite high reconfiguration time. The paper describes the basic runtime support as well as four optimizations necessary to achieve realtime performance given the limitations of DPR on today's FPGAs. The paper provides a characterization of a working runtime framework prototype on a Xilinx ZC706 development board. The paper also reports the performance of realtime computer vision pipelines when time-shared.