E2ATST: A Temporal-Spatial Optimized Energy-Efficient Architecture for Training Spiking Transformer
Yunhao Ma (1, 2), Yanyu Lin (1), Mingjing Li (1), Puli Quan (1), Chenlin Zhou (1), Wenyue Zhang (1, 2), Zhiwei Zhong (1), Wanyi Jia (1, 3, 4), Xueke Zhu (1), Qingyan Meng (1), Huihui Zhou (1, 3), Fengwei An (2)
TL;DR
This paper proposes E2ATST, a novel energy-efficient architecture for training spiking transformers that optimizes temporal and spatial aspects to reduce power consumption while maintaining performance.
Contribution
Introduces a new architecture that enhances energy efficiency in training spiking transformers through temporal-spatial optimization techniques.
Findings
Significant reduction in energy consumption during training.
Maintains comparable accuracy to traditional methods.
Improves training speed and efficiency.
Abstract
(1) Pengcheng Laboratory, (2) Southern University of Science and Technology, (3) Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, (4) University of Chinese Academy of Sciences
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Memory and Neural Computing · Parallel Computing and Optimization Techniques · Neural Networks and Reservoir Computing