HiNS: Hierarchical Negative Sampling for More Comprehensive Memory Retrieval Embedding Model

TL;DR

This paper introduces HiNS, a hierarchical negative sampling framework that improves memory retrieval in language agents by modeling negative sample difficulty and distribution, leading to better performance in memory-intensive tasks.

Contribution

HiNS is a novel data construction method that explicitly models negative sample difficulty tiers and ratios, enhancing embedding models' ability to discriminate in memory retrieval.

Findings

Significant performance improvements on LoCoMo and PERSONAMEM datasets.

Memory retrieval F1/BLEU-1 gains of over 3% on MemoryOS.

Total score improvements of over 2.5% on Mem0.

Abstract

Memory-augmented language agents rely on embedding models for effective memory retrieval. However, existing training data construction overlooks a critical limitation: the hierarchical difficulty of negative samples and their natural distribution in human-agent interactions. In practice, some negatives are semantically close distractors while others are trivially irrelevant, and natural dialogue exhibits structured proportions of these types. Current approaches using synthetic or uniformly sampled negatives fail to reflect this diversity, limiting embedding models' ability to learn nuanced discrimination essential for robust memory retrieval. In this work, we propose a principled data construction framework HiNS that explicitly models negative sample difficulty tiers and incorporates empirically grounded negative ratios derived from conversational data, enabling the training of embedding models with substantially improved retrieval fidelity and generalization in memory-intensive tasks. Experiments show significant improvements: on LoCoMo, F1/BLEU-1 gains of 3.27%/3.30%(MemoryOS) and 1.95%/1.78% (Mem0); on PERSONAMEM, total score improvements of 1.19% (MemoryOS) and 2.55% (Mem0).