Gen-AI for User Safety: A Survey

TL;DR

This survey reviews how Generative AI techniques are applied across various domains and data modalities to enhance user safety, addressing limitations of traditional ML classifiers in understanding natural language context.

Contribution

It provides the first comprehensive overview of Gen-AI applications for user safety across multiple domains and data types, including adversarial scenarios.

Findings

Gen-AI techniques are used in phishing, malware detection, content moderation, and physical safety.

Gen-AI effectively handles multi-modal data like text, images, videos, and audio.

Applications include adversarial settings to improve safety measures.

Abstract

Machine Learning and data mining techniques (i.e. supervised and unsupervised techniques) are used across domains to detect user safety violations. Examples include classifiers used to detect whether an email is spam or a web-page is requesting bank login information. However, existing ML/DM classifiers are limited in their ability to understand natural languages w.r.t the context and nuances. The aforementioned challenges are overcome with the arrival of Gen-AI techniques, along with their inherent ability w.r.t translation between languages, fine-tuning between various tasks and domains. In this manuscript, we provide a comprehensive overview of the various work done while using Gen-AI techniques w.r.t user safety. In particular, we first provide the various domains (e.g. phishing, malware, content moderation, counterfeit, physical safety) across which Gen-AI techniques have been applied. Next, we provide how Gen-AI techniques can be used in conjunction with various data modalities i.e. text, images, videos, audio, executable binaries to detect violations of user-safety. Further, also provide an overview of how Gen-AI techniques can be used in an adversarial setting. We believe that this work represents the first summarization of Gen-AI techniques for user-safety.