Lightweight Temporal Transformer Decomposition for Federated Autonomous Driving

TL;DR

This paper introduces a lightweight temporal transformer decomposition method for federated autonomous driving, effectively processing sequential data to improve robustness and real-time performance without resource-heavy fusion networks.

Contribution

It proposes a novel, resource-efficient transformer decomposition approach that enhances federated autonomous driving by leveraging temporal data while maintaining low complexity.

Findings

Outperforms recent methods on three datasets

Achieves real-time autonomous driving performance

Validated through real robot experiments

Abstract

Traditional vision-based autonomous driving systems often face difficulties in navigating complex environments when relying solely on single-image inputs. To overcome this limitation, incorporating temporal data such as past image frames or steering sequences, has proven effective in enhancing robustness and adaptability in challenging scenarios. While previous high-performance methods exist, they often rely on resource-intensive fusion networks, making them impractical for training and unsuitable for federated learning. To address these challenges, we propose lightweight temporal transformer decomposition, a method that processes sequential image frames and temporal steering data by breaking down large attention maps into smaller matrices. This approach reduces model complexity, enabling efficient weight updates for convergence and real-time predictions while leveraging temporal information to enhance autonomous driving performance. Intensive experiments on three datasets demonstrate that our method outperforms recent approaches by a clear margin while achieving real-time performance. Additionally, real robot experiments further confirm the effectiveness of our method.