Hierarchical Long-Term Semantic Memory for LinkedIn's Hiring Agent

TL;DR

The paper presents HLTM, a hierarchical semantic memory system for LLM agents, enhancing long-term knowledge management with scalability, privacy, and low-latency retrieval, demonstrated in LinkedIn's Hiring Assistant.

Contribution

Introduces HLTM, a schema-aligned hierarchical memory framework that improves scalability, privacy, and retrieval efficiency for LLM agents in industrial applications.

Findings

HLTM improves answer correctness and retrieval F1 by over 10%.

HLTM advances the latency-performance Pareto frontier.

HLTM is deployed in LinkedIn's Hiring Assistant for core features.

Abstract

Large Language Model (LLM) agents are increasingly used in real-world products, where personalized and context-aware user interactions are essential. A central enabler of such capabilities is the agent's long-term semantic memory system, which extracts implicit and explicit signals from noisy longitudinal behavioral data, stores them in a structured form, and supports low-latency retrieval. Building industrial-grade long-term memory for LLM agents raises five challenges: scalability, low-latency retrieval, privacy constraints, cross-domain generalizability, and observability. We introduce the Hierarchical Long-Term Semantic Memory (HLTM) framework, which organizes textual data into a schema-aligned memory tree that captures semantic knowledge at multiple levels of granularity, enabling scalable ingestion, privacy-aware storage, low-latency retrieval, and transparent provenance; HLTM further incorporates an adaptation mechanism to generalize across diverse use cases. Extensive evaluations on LinkedIn's Hiring Assistant show that HLTM improves answer correctness and retrieval F1 significantly by more than 10%, while significantly advancing the Pareto frontier between query and indexing latency. HLTM has been deployed in LinkedIn's Hiring Assistant to power core personalization features in production hiring workflows.