SecMate: Multi-Agent Adaptive Cybersecurity Troubleshooting with Tri-Context Personalization

TL;DR

SecMate is a multi-agent cybersecurity troubleshooting system that personalizes support using device, user, and service context, significantly improving resolution accuracy and user experience.

Contribution

It introduces a novel multi-agent framework integrating tri-context personalization and provides a comprehensive dataset and code for reproducible research.

Findings

Device-specific evidence increased correct resolutions from 50% to over 90%.

Step-by-step guidance improved user pleasantness and reduced burden.

Recommender system achieved high relevance with MRR@1=0.75.

Abstract

Recent advances in large language models and agentic frameworks have enabled virtual customer assistants (VCAs) for complex support. We present SecMate, a multi-agent VCA for cybersecurity troubleshooting that integrates device, user, and service specificity from conversational and device-level signals. Device specificity is provided by a lightweight local diagnostic utility, while user specificity relies on implicit proficiency inference and profile-aware troubleshooting. Service specificity is achieved through a proactive, context-aware recommender. We evaluate SecMate in a controlled study with 144 participants and 711 conversations. Device-level evidence increased correct resolutions from about 50% to over 90% relative to an LLM-only baseline, while step-by-step guidance improved pleasantness and reduced user burden. The recommender achieved high relevance (MRR@1=0.75), and participants showed strong willingness to substitute human IT support at costs well below human benchmarks. We release the full code base and a richly annotated dataset to support reproducible research on adaptive VCAs.