Practically implementing an LLM-supported collaborative vulnerability remediation process: a team-based approach

TL;DR

This paper presents a practical, team-based approach to integrating large language models into cybersecurity vulnerability remediation, emphasizing collaboration, role adaptation, and managing side effects based on real-world field studies.

Contribution

It introduces a novel, empirically validated collaborative process for LLM-supported vulnerability remediation, addressing practical challenges and stakeholder collaboration in cybersecurity.

Findings

Enhanced user satisfaction through collaborative LLM use

Effective role adaptation of LLMs based on task complexity

Identification of short-term side effects and mitigation strategies

Abstract

Incorporating LLM into cybersecurity operations, a typical real-world high-stakes task, is critical but non-trivial in practice. Using cybersecurity as the study context, we conduct a three-step mix-method study to incorporate LLM into the vulnerability remediation process effectively. Specifically, we deconstruct the deficiencies in user satisfaction within the existing process (Study 1). This inspires us to design, implement, and empirically validate an LLM-supported collaborative vulnerability remediation process through a field study (Study 2). Given LLM's diverse contributions, we further investigate LLM's double-edge roles through the analysis of remediation reports and follow-up interviews (Study 3). In essence, our contribution lies in promoting an efficient LLM-supported collaborative vulnerability remediation process. These first-hand, real-world pieces of evidence suggest that when incorporating LLMs into practical processes, facilitating the collaborations among all associated stakeholders, reshaping LLMs' roles according to task complexity, as well as approaching the short-term side effects of improved user engagement facilitated by LLMs with a rational mindset.