"I Always Felt that SomethingWasWrong.": Understanding Compliance Risks and Mitigation Strategies when Highly-Skilled Compliance Knowledge Workers Use Large Language Models

TL;DR

This study investigates compliance risks faced by highly-skilled knowledge workers using Large Language Models, revealing their mitigation strategies and the need for better guidance and organizational support.

Contribution

It provides empirical insights into the specific compliance challenges and mitigation practices of knowledge workers using LLMs, highlighting gaps in guidance and training.

Findings

Workers are concerned about sensitive data leakage.

Proactive risk mitigation strategies include data distortion and prompt limiting.

Lack of specific compliance guidance hampers risk management.

Abstract

The rapid advancement of Large Language Models (LLMs) has transformed knowledge-intensive has led to its widespread usage by knowledge workers to enhance their productivity. As these professionals handle sensitive information, and the training of text-based GenAI models involves the use of extensive data, there are thus concerns about privacy, security, and broader compliance with regulations and laws. While existing research has addressed privacy and security concerns, the specific compliance risks faced by highly-skilled knowledge workers when using the LLMs, and their mitigation strategies, remain underexplored. As understanding these risks and strategies is crucial for the development of industry-specific compliant LLM mechanisms, this research conducted semi-structured interviews with 24 knowledge workers from knowledge-intensive industries to understand their practices and experiences when integrating LLMs into their workflows. Our research explored how these workers ensure compliance and the resources and challenges they encounter when minimizing risks. Our preliminary findings showed that knowledge workers were concerned about the leakage of sensitive information and took proactive measures such as distorting input data and limiting prompt details to mitigate such risks. Their ability to identify and mitigate risks, however, was significantly hampered by a lack of LLM-specific compliance guidance and training. Our findings highlight the importance of improving knowledge workers' compliance awareness and establishing support systems and compliance cultures within organizations.