Generative Actor-Critic: An Off-policy Algorithm Using the Push-forward Model
Lingwei Peng, Hui Qian, Zhebang Shen, Chao Zhang, Fei Li

TL;DR
This paper introduces Generative Actor-Critic (GAC), a novel off-policy reinforcement learning algorithm that employs push-forward models to enhance policy expressiveness and exploration in continuous control tasks.
Contribution
The paper presents a density-free off-policy algorithm using push-forward models and an adaptive MMD-entropy regularizer, improving exploration and stability over traditional methods.
Findings
Push-forward policies enable multi-modality, enhancing exploration.
GAC achieves better asymptotic performance in continuous control tasks.
Adaptive regularization improves algorithm stability.
Abstract
Model-free deep reinforcement learning has achieved great success in many domains, such as video games, recommendation systems and robotic control tasks. In continuous control tasks, widely used policies with Gaussian distributions results in ineffective exploration of environments and limited performance of algorithms in many cases. In this paper, we propose a density-free off-policy algorithm, Generative Actor-Critic(GAC), using the push-forward model to increase the expressiveness of policies, which also includes an entropy-like technique, MMD-entropy regularizer, to balance the exploration and exploitation. Additionnally, we devise an adaptive mechanism to automatically scale this regularizer, which further improves the stability and robustness of GAC. The experiment results show that push-forward policies possess desirable features, such as multi-modality, which can improve the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Advanced Bandit Algorithms Research · Generative Adversarial Networks and Image Synthesis
