Computational Safety for Generative AI: A Signal Processing Perspective

TL;DR

This paper introduces a formal mathematical framework based on signal processing to assess and improve the safety of generative AI models, focusing on detecting malicious prompts and AI-generated content.

Contribution

It formalizes computational safety in GenAI using signal processing theory, providing methods for detecting malicious prompts and AI-generated outputs.

Findings

Sensitivity analysis can detect malicious prompts.

Signal processing techniques can identify AI-generated content.

Framework enables quantitative safety assessment.

Abstract

AI safety is a rapidly growing area of research that seeks to prevent the harm and misuse of frontier AI technology, particularly with respect to generative AI (GenAI) tools that are capable of creating realistic and high-quality content through text prompts. Examples of such tools include large language models (LLMs) and text-to-image (T2I) diffusion models. As the performance of various leading GenAI models approaches saturation due to similar training data sources and neural network architecture designs, the development of reliable safety guardrails has become a key differentiator for responsibility and sustainability. This paper presents a formalization of the concept of computational safety, which is a mathematical framework that enables the quantitative assessment, formulation, and study of safety challenges in GenAI through the lens of signal processing theory and methods. In particular, we explore two exemplary categories of computational safety challenges in GenAI that can be formulated as hypothesis testing problems. For the safety of model input, we show how sensitivity analysis and loss landscape analysis can be used to detect malicious prompts with jailbreak attempts. For the safety of model output, we elucidate how statistical signal processing and adversarial learning can be used to detect AI-generated content. Finally, we discuss key open research challenges, opportunities, and the essential role of signal processing in computational AI safety.