ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting

TL;DR

ScrapMem is a novel bio-inspired framework that enhances on-device multimodal memory efficiency and performance in LLM agents through optical forgetting and structured memory organization.

Contribution

It introduces Optical Forgetting for progressive memory compression and an Episodic Memory Graph to maintain semantic structure, achieving state-of-the-art results.

Findings

Achieved 51.0% Joint@10 score on ATM-Bench.

Reduced memory storage by up to 93%.

Increased Recall@10 to 70.3%.

Abstract

Long-term personalized memory for LLM agents is challenging on resource-limited edge devices due to high storage costs and multimodal complexity. To address this, we propose ScrapMem, a framework that integrates multimodal data into "Scrapbook Page." ScrapMem introduces Optical Forgetting, an optical compression mechanism that progressively reduces the resolution of older memories, lowering storage cost while suppressing low-value details. To maintain semantic consistency, we construct an Episodic Memory Graph (EM-Graph) that organizes key events into a causal-temporal structure. Extensive experiments on the multimodal ATM-Bench showcase that ScrapMem provides three main benefits: (1) strong performance, achieving a new state-of-the-art with a 51.0% Joint@10 score; (2) high storage efficiency, reducing memory usage by up to 93% via optical forgetting; and (3) improved recall, increasing Recall@10 to 70.3% through structured aggregation. ScrapMem offers an effective and storage-efficient solution for on-device long-term memory in multimodal LLM agents.