On the reproducibility of fully convolutional neural networks for modeling time-space evolving physical systems

TL;DR

This paper investigates the reproducibility issues of fully convolutional neural networks used for modeling time-space evolving physical systems, highlighting variability caused by non-deterministic GPU operations and the benefits of double precision training.

Contribution

It provides an empirical evaluation of how non-deterministic GPU operations affect the reproducibility of CNN models in physical system simulations and assesses precision improvements.

Findings

Significant variability in model weights and outputs due to GPU non-determinism.

Double precision training reduces variability and improves estimation accuracy.

Recurrent tasks amplify the effects of non-determinism.

Abstract

Reproducibility of a deep-learning fully convolutional neural network is evaluated by training several times the same network on identical conditions (database, hyperparameters, hardware) with non-deterministic Graphics Processings Unit (GPU) operations. The propagation of two-dimensional acoustic waves, typical of time-space evolving physical systems, is studied on both recursive and non-recursive tasks. Significant changes in models properties (weights, featured fields) are observed. When tested on various propagation benchmarks, these models systematically returned estimations with a high level of deviation, especially for the recurrent analysis which strongly amplifies variability due to the non-determinism. Trainings performed with double floating-point precision provide slightly better estimations and a significant reduction of the variability of both the network parameters and its testing error range.