Generative Adversarial Simulator
Jonathan Raiman

TL;DR
This paper introduces a novel simulator-free knowledge distillation method for reinforcement learning policies, using adversarial training to generate multiple exemplars per class, improving data-free learning on benchmark datasets.
Contribution
It presents the first simulator-free policy distillation approach that explicitly handles multiple observations per output class using adversarial loss.
Findings
Outperforms state-of-the-art data-free learning on MNIST, Fashion-MNIST, CIFAR-10
Effectively addresses multiple input modes in data generation
Identifies challenges in high-dimensional environments like Pong and Seaquest
Abstract
Knowledge distillation between machine learning models has opened many new avenues for parameter count reduction, performance improvements, or amortizing training time when changing architectures between the teacher and student network. In the case of reinforcement learning, this technique has also been applied to distill teacher policies to students. Until now, policy distillation required access to a simulator or real world trajectories. In this paper we introduce a simulator-free approach to knowledge distillation in the context of reinforcement learning. A key challenge is having the student learn the multiplicity of cases that correspond to a given action. While prior work has shown that data-free knowledge distillation is possible with supervised learning models by generating synthetic examples, these approaches to are vulnerable to only producing a single prototype example for…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Model Reduction and Neural Networks · Explainable Artificial Intelligence (XAI)
MethodsKnowledge Distillation
