Logits are predictive of network type

TL;DR

This paper demonstrates that it is possible to accurately predict which specific deep neural network generated a given logit vector, revealing insights into network encoding and potential applications in security and model understanding.

Contribution

The study introduces a method to classify network types based on logit vectors, showing high accuracy and generalization across different training conditions.

Findings

High classification accuracy for network identification

Unnormalized logits yield better results than normalized ones

Limited transferability across different weight sets

Abstract

We show that it is possible to predict which deep network has generated a given logit vector with accuracy well above chance. We utilize a number of networks on a dataset, initialized with random weights or pretrained weights, as well as fine-tuned networks. A classifier is then trained on the logit vectors of the trained set of this dataset to map the logit vector to the network index that has generated it. The classifier is then evaluated on the test set of the dataset. Results are better with randomly initialized networks, but also generalize to pretrained networks as well as fine-tuned ones. Classification accuracy is higher using unnormalized logits than normalized ones. We find that there is little transfer when applying a classifier to the same networks but with different sets of weights. In addition to help better understand deep networks and the way they encode uncertainty, we anticipate our finding to be useful in some applications (e.g. tailoring an adversarial attack for a certain type of network). Code is available at https://github.com/aliborji/logits.