Open Problems and Modern Solutions for Deep Reinforcement Learning

TL;DR

This paper reviews recent solutions addressing key challenges in Deep Reinforcement Learning, focusing on reward design and feature attention mechanisms to improve efficiency, flexibility, and performance.

Contribution

It highlights two innovative approaches: one combining extrinsic and intrinsic rewards for better task performance, and another using selective attention and particle filters for efficient feature selection.

Findings

Enhanced obstacle avoidance through combined reward design

Improved efficiency and flexibility via attention and particle filters

Discussion of future research directions in DRL

Abstract

Deep Reinforcement Learning (DRL) has achieved great success in solving complicated decision-making problems. Despite the successes, DRL is frequently criticized for many reasons, e.g., data inefficient, inflexible and intractable reward design. In this paper, we review two publications that investigate the mentioned issues of DRL and propose effective solutions. One designs the reward for human-robot collaboration by combining the manually designed extrinsic reward with a parameterized intrinsic reward function via the deterministic policy gradient, which improves the task performance and guarantees a stronger obstacle avoidance. The other one applies selective attention and particle filters to rapidly and flexibly attend to and select crucial pre-learned features for DRL using approximate inference instead of backpropagation, thereby improving the efficiency and flexibility of DRL. Potential avenues for future work in both domains are discussed in this paper.