Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making
Jeonghye Kim, Suyoung Lee, Woojun Kim, Youngchul Sung

TL;DR
Decision ConvFormer introduces local convolution filtering within a MetaFormer architecture to better capture local dependencies in RL trajectories, achieving state-of-the-art results with improved generalization and resource efficiency.
Contribution
The paper presents Decision ConvFormer, a novel architecture that replaces attention with local convolution filtering, effectively modeling local dependencies in RL data.
Findings
Achieved state-of-the-art performance on standard RL benchmarks.
Requires fewer computational resources than existing models.
Demonstrated improved understanding and generalization in RL tasks.
Abstract
The recent success of Transformer in natural language processing has sparked its use in various domains. In offline reinforcement learning (RL), Decision Transformer (DT) is emerging as a promising model based on Transformer. However, we discovered that the attention module of DT is not appropriate to capture the inherent local dependence pattern in trajectories of RL modeled as a Markov decision process. To overcome the limitations of DT, we propose a novel action sequence predictor, named Decision ConvFormer (DC), based on the architecture of MetaFormer, which is a general structure to process multiple entities in parallel and understand the interrelationship among the multiple entities. DC employs local convolution filtering as the token mixer and can effectively capture the inherent local associations of the RL dataset. In extensive experiments, DC achieved state-of-the-art…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsTopic Modeling · Machine Learning and Data Classification · Data Stream Mining Techniques
MethodsMulti-Head Attention · Convolution · Dense Connections · Linear Layer · Label Smoothing · Absolute Position Encodings · Attention Is All You Need · Adam · MetaFormer · Residual Connection
