Fake or Real: The Impostor Hunt in Texts for Space Operations

TL;DR

This paper discusses a Kaggle competition focused on detecting malicious modifications in Large Language Model outputs, addressing AI security threats like data poisoning and overreliance in space-related applications.

Contribution

It introduces a novel challenge in AI security for space operations, encouraging development of new detection techniques for LLM output manipulation.

Findings

Participants developed new methods for detecting malicious LLM outputs

The competition highlighted the importance of AI security in space applications

Initial solutions showed promising results in identifying impostor texts

Abstract

The "Fake or Real" competition hosted on Kaggle (https://www.kaggle.com/competitions/fake-or-real-the-impostor-hunt ) is the second part of a series of follow-up competitions and hackathons related to the "Assurance for Space Domain AI Applications" project funded by the European Space Agency (https://assurance-ai.space-codev.org/ ). The competition idea is based on two real-life AI security threats identified within the project -- data poisoning and overreliance in Large Language Models. The task is to distinguish between the proper output from LLM and the output generated under malicious modification of the LLM. As this problem was not extensively researched, participants are required to develop new techniques to address this issue or adjust already existing ones to this problem's statement.