Comparative Insights on Adversarial Machine Learning from Industry and Academia: A User-Study Approach

TL;DR

This paper explores industry and academic perspectives on adversarial machine learning through surveys and challenges, highlighting the importance of security education in ML curricula.

Contribution

It presents two studies: an industry survey linking cybersecurity education to AML concerns, and a student engagement assessment using CTF challenges with NLP and Generative AI.

Findings

Cybersecurity education correlates with AML threat concern.

CTF challenges effectively engage students in AML topics.

Participants recommend integrating security into ML education.

Abstract

An exponential growth of Machine Learning and its Generative AI applications brings with it significant security challenges, often referred to as Adversarial Machine Learning (AML). In this paper, we conducted two comprehensive studies to explore the perspectives of industry professionals and students on different AML vulnerabilities and their educational strategies. In our first study, we conducted an online survey with professionals revealing a notable correlation between cybersecurity education and concern for AML threats. For our second study, we developed two CTF challenges that implement Natural Language Processing and Generative AI concepts and demonstrate a poisoning attack on the training data set. The effectiveness of these challenges was evaluated by surveying undergraduate and graduate students at Carnegie Mellon University, finding that a CTF-based approach effectively engages interest in AML threats. Based on the responses of the participants in our research, we provide detailed recommendations emphasizing the critical need for integrated security education within the ML curriculum.