Engagement Undermines Safety: How Stereotypes and Toxicity Shape Humor in Language Models

TL;DR

This study investigates how humor generation in large language models can inadvertently reinforce stereotypes and toxicity, revealing biases and structural issues in humor and safety alignment.

Contribution

It introduces a comprehensive evaluation framework linking humor, stereotypes, and toxicity in LLMs, highlighting bias amplification and structural embedding of harmful content.

Findings

Harmful outputs receive higher humor scores, especially with role-based prompts.

Harmful cues increase predictive uncertainty and can make harmful punchlines more expected.

Satire generation in LLMs increases stereotypicality and toxicity, affecting human perceptions.

Abstract

Large language models are increasingly used for creative writing and engagement content, raising safety concerns about the outputs. Therefore, casting humor generation as a testbed, this work evaluates how funniness optimization in modern LLM pipelines couples with harmful content by jointly measuring humor, stereotypicality, and toxicity. This is further supplemented by analyzing incongruity signals through information-theoretic metrics. Across six models, we observe that harmful outputs receive higher humor scores which further increase under role-based prompting, indicating a bias amplification loop between generators and evaluators. Information-theoretic analyses show harmful cues widen predictive uncertainty and surprisingly, can even make harmful punchlines more expected for some models, suggesting structural embedding in learned humor distributions. External validation on an additional satire-generation task with human perceived funniness judgments shows that LLM satire increases stereotypicality and typically toxicity, including for closed models. Quantitatively, stereotypical/toxic jokes gain $10-21\%$ in mean humor score, stereotypical jokes appear $11\%$ to $28\%$ more often among the jokes marked funny by LLM-based metric and up to $10\%$ more often in generations perceived as funny by humans.