MCRM: Mother Compact Recurrent Memory
Abduallah A. Mohamed, Christian Claudel

TL;DR
This paper introduces MCRM, a nested LSTM-GRU architecture that enhances memory capabilities by combining long-term and short-term neuron functions, showing improved performance on specific tasks.
Contribution
The paper proposes MCRM, a novel nested LSTM-GRU architecture with a compact memory pattern, integrating forget and input gates as input to the GRU, advancing recurrent neural network design.
Findings
MCRM outperforms previous architectures on certain tasks.
MCRM has a compact memory pattern with neurons functioning in both long-term and short-term modes.
Empirical results demonstrate the effectiveness of MCRM in sequence modeling.
Abstract
LSTMs and GRUs are the most common recurrent neural network architectures used to solve temporal sequence problems. The two architectures have differing data flows dealing with a common component called the cell state (also referred to as the memory). We attempt to enhance the memory by presenting a modification that we call the Mother Compact Recurrent Memory (MCRM). MCRMs are a type of a nested LSTM-GRU architecture where the cell state is the GRU hidden state. The concatenation of the forget gate and input gate interactions from the LSTM are considered an input to the GRU cell. Because MCRMs has this type of nesting, MCRMs have a compact memory pattern consisting of neurons that acts explicitly in both long-term and short-term fashions. For some specific tasks, empirical results show that MCRMs outperform previously used architectures.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Neural Networks and Reservoir Computing · Reinforcement Learning in Robotics
MethodsSigmoid Activation · Tanh Activation · Gated Recurrent Unit · Long Short-Term Memory
