LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centres

TL;DR

This study empirically examines how SOC analysts use LLMs over 10 months, revealing they serve as on-demand aids for sensemaking and technical interpretation, supporting human decision-making without replacing analysts.

Contribution

It provides the first longitudinal analysis of real-world SOC analyst interactions with LLMs, highlighting their role as cognitive aids and offering design insights for human-centered AI in cybersecurity.

Findings

LLMs are used mainly for sensemaking and context-building.

93% of queries align with established cybersecurity competencies.

Usage shifts from exploration to routine integration over time.

Abstract

The integration of Large Language Models (LLMs) into Security Operations Centres (SOCs) presents a transformative, yet still evolving, opportunity to reduce analyst workload through human-AI collaboration. However, their real-world application in SOCs remains underexplored. To address this gap, we present a longitudinal study of 3,090 analyst queries from 45 SOC analysts over 10 months. Our analysis reveals that analysts use LLMs as on-demand aids for sensemaking and context-building, rather than for making high-stakes determinations, preserving analyst decision authority. The majority of queries are related to interpreting low-level telemetry (e.g., commands) and refining technical communication through short (1-3 turn) interactions. Notably, 93% of queries align with established cybersecurity competencies (NICE Framework), underscoring the relevance of LLM use for SOC-related tasks. Despite variations in tasks and engagement, usage trends indicate a shift from occasional exploration to routine integration, with growing adoption and sustained use among a subset of analysts. We find that LLMs function as flexible, on-demand cognitive aids that augment, rather than replace, SOC expertise. Our study provides actionable guidance for designing context-aware, human-centred AI assistance in security operations, highlighting the need for further in-the-wild research on real-world analyst-LLM collaboration, challenges, and impacts.