Reducing the Deployment-Time Inference Control Costs of Deep Reinforcement Learning Agents via an Asymmetric Architecture

TL;DR

This paper introduces an asymmetric neural network architecture for deep reinforcement learning that switches between a high-cost and low-cost policy to reduce inference energy consumption on mobile robots without sacrificing performance.

Contribution

The paper proposes a novel asymmetric architecture that dynamically switches between policies to lower inference costs in DRL, suitable for energy-constrained platforms.

Findings

Significant reduction in inference costs demonstrated on robotic control benchmarks.

Maintains comparable performance to traditional DRL methods.

Applicable to energy-limited robotic systems.

Abstract

Deep reinforcement learning (DRL) has been demonstrated to provide promising results in several challenging decision making and control tasks. However, the required inference costs of deep neural networks (DNNs) could prevent DRL from being applied to mobile robots which cannot afford high energy-consuming computations. To enable DRL methods to be affordable in such energy-limited platforms, we propose an asymmetric architecture that reduces the overall inference costs via switching between a computationally expensive policy and an economic one. The experimental results evaluated on a number of representative benchmark suites for robotic control tasks demonstrate that our method is able to reduce the inference costs while retaining the agent's overall performance.