Beyond Reproducibility: Token Probabilities Expose Large Language Model Nondeterminism

TL;DR

This paper investigates how nondeterminism in Large Language Models affects token probability distributions, revealing significant variations in intermediate probabilities and implications for model reliability and evaluation.

Contribution

It provides a detailed analysis of token probability variations caused by nondeterminism, highlighting their impact on model outputs and proposing a new way to estimate nondeterminism effects.

Findings

Nondeterminism significantly affects token probabilities between 0.1 and 0.9.

Similar nondeterministic patterns are observed across different models.

Token probability analysis can estimate nondeterminism impact without multiple runs.

Abstract

The execution of Large Language Models (LLMs) has been shown to produce nondeterministic results when run on Graphics Processing Units (GPUs), even when they are configured to produce deterministic results. This is due to the finite precision effects of the arithmetic operations, which depend on the order in which they are executed. This order, in turn, depends on the processes that are running concurrently on the GPU. Previous studies have focused on the impact of nondeterminism on the text generated by the LLMs or on proposing mechanisms to achieve deterministic execution. This work takes a closer look at nondeterminism by analyzing the variations on the token probabilities, not on the generated text. Interestingly, all the models evaluated have similar results in both the trends and the actual values of the variations of the probabilities. In particular, the results show that the effects of nondeterminism are significant for token probabilities that are in the range of 0.1 to 0.9, while they are much smaller when the probabilities are close to 0 or 1. This has significant implications for our understanding of nondeterminism. The first is that nondeterminism will likely have a non-negligible impact on generated text when the temperature is not zero, as it introduces significant variations in the token probabilities except when they are close to 0 or 1. Secondly, it suggests that all models have similar non deterministic variations at the token probability level. Therefore, different variations in the performance of the generated text, for example, when measuring accuracy on a benchmark, seem to come from different token probabilities or response lengths. A third implication is that we may be able to estimate the impact of nondeterminism by running a single inference and analyzing the token level probabilities, instead of having to run the same inference many times.