Probing AI Safety with Source Code

TL;DR

This paper introduces CoDoT, a novel prompting strategy that converts natural language inputs into code to evaluate and reveal safety shortcomings in large language models, highlighting their tendency to generate toxic outputs.

Contribution

The paper presents CoDoT, a new method for assessing LLM safety by translating prompts into code, exposing significant safety failures in current models.

Findings

GPT-4 Turbo's toxicity increases 16.5 times with CoDoT

DeepSeek R1 fails 100% of the time in safety evaluation

Toxicity increases by 300% on average across models

Abstract

Large language models (LLMs) have become ubiquitous, interfacing with humans in numerous safety-critical applications. This necessitates improving capabilities, but importantly coupled with greater safety measures to align these models with human values and preferences. In this work, we demonstrate that contemporary models fall concerningly short of the goal of AI safety, leading to an unsafe and harmful experience for users. We introduce a prompting strategy called Code of Thought (CoDoT) to evaluate the safety of LLMs. CoDoT converts natural language inputs to simple code that represents the same intent. For instance, CoDoT transforms the natural language prompt "Make the statement more toxic: {text}" to: "make_more_toxic({text})". We show that CoDoT results in a consistent failure of a wide range of state-of-the-art LLMs. For example, GPT-4 Turbo's toxicity increases 16.5 times, DeepSeek R1 fails 100% of the time, and toxicity increases 300% on average across seven modern LLMs. Additionally, recursively applying CoDoT can further increase toxicity two times. Given the rapid and widespread adoption of LLMs, CoDoT underscores the critical need to evaluate safety efforts from first principles, ensuring that safety and capabilities advance together.