Towards real-time and energy efficient Siamese tracking -- a hardware-software approach

TL;DR

This paper presents a hardware-software co-designed, energy-efficient Siamese tracker implementation on FPGA that achieves real-time performance suitable for low-power edge devices, maintaining accuracy comparable to high-end GPU solutions.

Contribution

It introduces a quantised Siamese network optimized for FPGA acceleration, enabling real-time tracking with high energy efficiency on embedded devices.

Findings

Achieved nearly 50 fps with FPGA-accelerated Siamese network.

Maintained tracker accuracy comparable to floating point SiamFC.

System runs at up to 17 fps on ARM+FPGA platform, suitable for low-power applications.

Abstract

Siamese trackers have been among the state-of-the-art solutions in each Visual Object Tracking (VOT) challenge over the past few years. However, with great accuracy comes great computational complexity: to achieve real-time processing, these trackers have to be massively parallelised and are usually run on high-end GPUs. Easy to implement, this approach is energy consuming, and thus cannot be used in many low-power applications. To overcome this, one can use energy-efficient embedded devices, such as heterogeneous platforms joining the ARM processor system with programmable logic (FPGA). In this work, we propose a hardware-software implementation of the well-known fully connected Siamese tracker (SiamFC). We have developed a quantised Siamese network for the FINN accelerator, using algorithm-accelerator co-design, and performed design space exploration to achieve the best efficiency-to-energy ratio (determined by FPS and used resources). For our network, running in the programmable logic part of the Zynq UltraScale+ MPSoC ZCU104, we achieved the processing of almost 50 frames-per-second with tracker accuracy on par with its floating point counterpart, as well as the original SiamFC network. The complete tracking system, implemented in ARM with the network accelerated on FPGA, achieves up to 17 fps. These results bring us towards bridging the gap between the highly accurate but energy-demanding algorithms and energy-efficient solutions ready to be used in low-power, edge systems.