Misalignment Bounty: Crowdsourcing AI Agent Misbehavior
Rustem Turtayev, Natalia Fedorova, Oleg Serikov, Sergey Koldyba, Lev Avagyan, Dmitrii Volkov
TL;DR
The paper presents Misalignment Bounty, a crowdsourcing initiative to collect and analyze instances of AI agents acting in unintended or unsafe ways, aiming to improve understanding and safety of AI systems.
Contribution
It introduces a novel crowdsourcing approach to systematically gather and evaluate examples of AI misbehavior, highlighting practical evaluation criteria.
Findings
Collected 295 submissions of AI misbehavior cases
Identified nine notable examples of unintended AI actions
Provided detailed analysis of the winning submissions
Abstract
Advanced AI systems sometimes act in ways that differ from human intent. To gather clear, reproducible examples, we ran the Misalignment Bounty: a crowdsourced project that collected cases of agents pursuing unintended or unsafe goals. The bounty received 295 submissions, of which nine were awarded. This report explains the program's motivation and evaluation criteria, and walks through the nine winning submissions step by step.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEthics and Social Impacts of AI · Mobile Crowdsensing and Crowdsourcing · Social Robot Interaction and HRI