Large Language Models as a (Bad) Security Norm in the Context of Regulation and Compliance

TL;DR

This paper examines how Large Language Models (LLMs) pose security and legal risks in cybersecurity, highlighting their limitations and proposing solutions to mitigate negative impacts on security practices and compliance.

Contribution

It provides an analysis of the security and legal challenges posed by LLMs in cybersecurity, and suggests practical and regulatory solutions to address these issues.

Findings

LLMs often fail to meet cybersecurity legal obligations.

Using LLMs can undermine security best practices.

Replacing LLMs with symbolic AI can mitigate risks.

Abstract

The use of Large Language Models (LLM) by providers of cybersecurity and digital infrastructures of all kinds is an ongoing development. It is suggested and on an experimental basis used to write the code for the systems, and potentially fed with sensitive data or what would otherwise be considered trade secrets. Outside of these obvious points, this paper asks how AI can negatively affect cybersecurity and law when used for the design and deployment of security infrastructure by its developers. Firstly, the paper discusses the use of LLMs in security, either directly or indirectly, and briefly tackles other types of AI. It then lists norms in cybersecurity, then a range of legal cybersecurity obligations from the European Union, to create a frame of reference. Secondly, the paper describes how LLMs may fail to fulfil both legal obligations and best practice in cybersecurity is given, and the paper ends with some economic and practical consequences for this development, with some notions of solutions as well. The paper finds that using LLMs comes with many risks, many of which are against good security practice, and the legal obligations in security regulation. This is because of the inherent weaknesses of LLMs, most of which are mitigated if replaced with symbolic AI. Both also have issues fulfilling basic traceability obligations and practice. Solutions are secondary systems surrounding LLM based AI, fulfilment of security norms beyond legal requirements and simply not using such technology in certain situations.