SpikeTrack: A Spike-driven Framework for Efficient Visual Tracking

TL;DR

SpikeTrack introduces a novel spike-driven framework for RGB object tracking that significantly improves energy efficiency while maintaining high accuracy, leveraging spatiotemporal dynamics and a memory-retrieval module.

Contribution

It is the first spike-driven framework for RGB tracking that balances energy efficiency and accuracy through asymmetric design and neural-inspired memory retrieval.

Findings

Achieves state-of-the-art results among SNN trackers.

Surpasses TransT on LaSOT dataset.

Consumes only 1/26 of TransT's energy.

Abstract

Spiking Neural Networks (SNNs) promise energy-efficient vision, but applying them to RGB visual tracking remains difficult: Existing SNN tracking frameworks either do not fully align with spike-driven computation or do not fully leverage neurons' spatiotemporal dynamics, leading to a trade-off between efficiency and accuracy. To address this, we introduce SpikeTrack, a spike-driven framework for energy-efficient RGB object tracking. SpikeTrack employs a novel asymmetric design that uses asymmetric timestep expansion and unidirectional information flow, harnessing spatiotemporal dynamics while cutting computation. To ensure effective unidirectional information transfer between branches, we design a memory-retrieval module inspired by neural inference mechanisms. This module recurrently queries a compact memory initialized by the template to retrieve target cues and sharpen target perception over time. Extensive experiments demonstrate that SpikeTrack achieves the state-of-the-art among SNN-based trackers and remains competitive with advanced ANN trackers. Notably, it surpasses TransT on LaSOT dataset while consuming only 1/26 of its energy. To our knowledge, SpikeTrack is the first spike-driven framework to make RGB tracking both accurate and energy efficient. The code and models are available at https://github.com/faicaiwawa/SpikeTrack.