TMLab: Generative Enhanced Model (GEM) for adversarial attacks

TL;DR

GEM is a generative model based on GPT-2 that produces controlled, natural-like English sentences for adversarial attacks, successfully creating misleading claims that challenge fact-checking.

Contribution

The paper introduces GEM, a novel extension of GPT-2 with target vocabulary input for controlled text generation in adversarial attack scenarios.

Findings

GEM won first prize on FEVER 2.0 Breakers Task.

GEM can generate misleading claims by mixing facts from different sources.

GEM demonstrates effective controlled text generation for adversarial purposes.

Abstract

We present our Generative Enhanced Model (GEM) that we used to create samples awarded the first prize on the FEVER 2.0 Breakers Task. GEM is the extended language model developed upon GPT-2 architecture. The addition of novel target vocabulary input to the already existing context input enabled controlled text generation. The training procedure resulted in creating a model that inherited the knowledge of pretrained GPT-2, and therefore was ready to generate natural-like English sentences in the task domain with some additional control. As a result, GEM generated malicious claims that mixed facts from various articles, so it became difficult to classify their truthfulness.