Improved Memories Learning
Francesco Varoli, Guido Novati, Pantelis R. Vlachas, Petros, Koumoutsakos

TL;DR
Improved Memories Learning (IMeL) transforms reinforcement learning into a supervised learning task by using a reservoir of experiences and a neural network that interpolates based on experience similarity, simplifying policy learning.
Contribution
IMeL introduces a novel RL algorithm that leverages experience reservoirs and neural network interpolation, emphasizing non-parametric policy improvements over traditional neural encoding.
Findings
Preliminary results demonstrate the effectiveness of IMeL.
IMeL provides a simple baseline for evaluating complex models.
The method emphasizes interpolation over neural encoding.
Abstract
We propose Improved Memories Learning (IMeL), a novel algorithm that turns reinforcement learning (RL) into a supervised learning (SL) problem and delimits the role of neural networks (NN) to interpolation. IMeL consists of two components. The first is a reservoir of experiences. Each experience is updated based on a non-parametric procedural improvement of the policy, computed as a bounded one-sample Monte Carlo estimate. The second is a NN regressor, which receives as input improved experiences from the reservoir (context points) and computes the policy by interpolation. The NN learns to measure the similarity between states in order to compute long-term forecasts by averaging experiences, rather than by encoding the problem structure in the NN parameters. We present preliminary results and propose IMeL as a baseline method for assessing the merits of more complex models and inductive…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Machine Learning and Algorithms · Domain Adaptation and Few-Shot Learning
