Predicting Generalization in Deep Learning via Metric Learning -- PGDL Shared task

TL;DR

This paper presents a solution for the PGDL competition that predicts deep learning model generalization by combining simple metrics of neural network properties through automated testing.

Contribution

It introduces a method that uses combined simple metrics and automatic testing to predict model generalization, advancing understanding of neural network behavior.

Findings

Achieved eighth place in the PGDL competition.

Demonstrated effectiveness of metric combination for generalization prediction.

Explored various neural network properties for improved predictions.

Abstract

The competition "Predicting Generalization in Deep Learning (PGDL)" aims to provide a platform for rigorous study of generalization of deep learning models and offer insight into the progress of understanding and explaining these models. This report presents the solution that was submitted by the user \emph{smeznar} which achieved the eight place in the competition. In the proposed approach, we create simple metrics and find their best combination with automatic testing on the provided dataset, exploring how combinations of various properties of the input neural network architectures can be used for the prediction of their generalization.