Learning Computational Efficient Bots with Costly Features

TL;DR

This paper introduces a cost-aware decision transformer that dynamically selects features to balance computational efficiency and task performance in real-time decision-making scenarios.

Contribution

It proposes the Budgeted Decision Transformer, extending the Decision Transformer to incorporate feature cost constraints for improved efficiency.

Findings

Achieves comparable performance with fewer computational resources.

Demonstrates effectiveness on D4RL benchmarks and 3D video game environments.

Reduces inference time by dynamically selecting features.

Abstract

Deep reinforcement learning (DRL) techniques have become increasingly used in various fields for decision-making processes. However, a challenge that often arises is the trade-off between both the computational efficiency of the decision-making process and the ability of the learned agent to solve a particular task. This is particularly critical in real-time settings such as video games where the agent needs to take relevant decisions at a very high frequency, with a very limited inference time. In this work, we propose a generic offline learning approach where the computation cost of the input features is taken into account. We derive the Budgeted Decision Transformer as an extension of the Decision Transformer that incorporates cost constraints to limit its cost at inference. As a result, the model can dynamically choose the best input features at each timestep. We demonstrate the effectiveness of our method on several tasks, including D4RL benchmarks and complex 3D environments similar to those found in video games, and show that it can achieve similar performance while using significantly fewer computational resources compared to classical approaches.