Revolutionizing Long-Term Memory in AI: New Horizons with High-Capacity and High-Speed Storage

TL;DR

This paper explores alternative memory storage approaches for AI, emphasizing retaining raw experiences for flexible, on-demand retrieval to enhance long-term memory and avoid information loss, supported by preliminary experiments.

Contribution

It introduces the 'store then on-demand extract' approach and discusses methods to improve experience sharing and insight discovery in AI memory systems.

Findings

Preliminary experiments support the effectiveness of the proposed approaches.

Retaining raw experiences can prevent information loss compared to extraction-based methods.

Sharing stored experiences improves collection efficiency.

Abstract

Driven by our mission of "uplifting the world with memory," this paper explores the design concept of "memory" that is essential for achieving artificial superintelligence (ASI). Rather than proposing novel methods, we focus on several alternative approaches whose potential benefits are widely imaginable, yet have remained largely unexplored. The currently dominant paradigm, which can be termed "extract then store," involves extracting information judged to be useful from experiences and saving only the extracted content. However, this approach inherently risks the loss of information, as some valuable knowledge particularly for different tasks may be discarded in the extraction process. In contrast, we emphasize the "store then on-demand extract" approach, which seeks to retain raw experiences and flexibly apply them to various tasks as needed, thus avoiding such information loss. In addition, we highlight two further approaches: discovering deeper insights from large collections of probabilistic experiences, and improving experience collection efficiency by sharing stored experiences. While these approaches seem intuitively effective, our simple experiments demonstrate that this is indeed the case. Finally, we discuss major challenges that have limited investigation into these promising directions and propose research topics to address them.