Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture
Sangjun Park, JinYeong Bak

TL;DR
Memoria is a human-inspired memory system for neural networks that effectively addresses long-term forgetting, outperforming traditional methods across various tasks by mimicking key human memory effects.
Contribution
The paper introduces Memoria, a novel memory architecture inspired by human neuroscience, to improve long-term retention in neural networks.
Findings
Memoria outperforms conventional memory techniques in sorting, language modeling, and classification.
It exhibits primacy, recency, and temporal contiguity effects similar to human memory.
Experimental results demonstrate its effectiveness across diverse tasks.
Abstract
Making neural networks remember over the long term has been a longstanding issue. Although several external memory techniques have been introduced, most focus on retaining recent information in the short term. Regardless of its importance, information tends to be fatefully forgotten over time. We present Memoria, a memory system for artificial neural networks, drawing inspiration from humans and applying various neuroscientific and psychological theories. The experimental results prove the effectiveness of Memoria in the diverse tasks of sorting, language modeling, and classification, surpassing conventional techniques. Engram analysis reveals that Memoria exhibits the primacy, recency, and temporal contiguity effects which are characteristics of human memory.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobotics and Automated Systems · Context-Aware Activity Recognition Systems · Parallel Computing and Optimization Techniques
MethodsFocus · Multi-Head Attention · Attention Is All You Need · Cosine Annealing · Linear Warmup With Cosine Annealing · Dropout · Byte Pair Encoding · Discriminative Fine-Tuning · WordPiece · Attention Dropout
