Odyssey: Creation, Analysis and Detection of Trojan Models

TL;DR

This paper introduces Odysseus, a comprehensive dataset and a novel detection method that analyzes intrinsic DNN properties to identify Trojan attacks without prior knowledge of triggers, significantly improving detection accuracy.

Contribution

The paper presents Odysseus, the largest diverse Trojan model dataset, and a new detection approach based on intrinsic DNN properties affected by Trojaning, outperforming existing methods.

Findings

Trojan attacks alter classifier margin and decision boundary shape.

The proposed detector outperforms existing methods across various models.

Effective detection on unseen triggers and regularized models.

Abstract

Along with the success of deep neural network (DNN) models, rise the threats to the integrity of these models. A recent threat is the Trojan attack where an attacker interferes with the training pipeline by inserting triggers into some of the training samples and trains the model to act maliciously only for samples that contain the trigger. Since the knowledge of triggers is privy to the attacker, detection of Trojan networks is challenging. Existing Trojan detectors make strong assumptions about the types of triggers and attacks. We propose a detector that is based on the analysis of the intrinsic DNN properties; that are affected due to the Trojaning process. For a comprehensive analysis, we develop Odysseus, the most diverse dataset to date with over 3,000 clean and Trojan models. Odysseus covers a large spectrum of attacks; generated by leveraging the versatility in trigger designs and source to target class mappings. Our analysis results show that Trojan attacks affect the classifier margin and shape of decision boundary around the manifold of clean data. Exploiting these two factors, we propose an efficient Trojan detector that operates without any knowledge of the attack and significantly outperforms existing methods. Through a comprehensive set of experiments we demonstrate the efficacy of the detector on cross model architectures, unseen Triggers and regularized models.