MemCollab: Cross-Agent Memory Collaboration via Contrastive Trajectory Distillation

TL;DR

MemCollab introduces a contrastive trajectory distillation method to create shared, agent-agnostic memory for diverse LLM agents, improving reasoning accuracy and efficiency by capturing shared invariants and suppressing agent-specific biases.

Contribution

The paper proposes MemCollab, a novel framework that constructs shared memory across heterogeneous agents using contrastive learning to enhance reasoning capabilities.

Findings

Improves accuracy on mathematical reasoning and code generation benchmarks.

Enhances inference efficiency across diverse agent types.

Successfully enables shared reasoning resources for heterogeneous LLM agents.

Abstract

Large language model (LLM)-based agents rely on memory mechanisms to reuse knowledge from past problem-solving experiences. Existing approaches typically construct memory in a per-agent manner, tightly coupling stored knowledge to a single model's reasoning style. In modern deployments with heterogeneous agents, a natural question arises: can a single memory system be shared across different models? We found that naively transferring memory between agents often degrades performance, as such memory entangles task-relevant knowledge with agent-specific biases. To address this challenge, we propose MemCollab, a collaborative memory framework that constructs agent-agnostic memory by contrasting reasoning trajectories generated by different agents on the same task. This contrastive process distills abstract reasoning constraints that capture shared task-level invariants while suppressing agent-specific artifacts. We further introduce a task-aware retrieval mechanism that conditions memory access on task category, ensuring that only relevant constraints are used at inference time. Experiments on mathematical reasoning and code generation benchmarks demonstrate that MemCollab consistently improves both accuracy and inference-time efficiency across diverse agents, including cross-modal-family settings. Our results show that the collaboratively constructed memory can function as a shared reasoning resource for diverse LLM-based agents.