Maximum Spanning Trees Are Invariant to Temperature Scaling in Graph-based Dependency Parsing

TL;DR

This paper proves that temperature scaling, a common calibration method, does not affect the output of maximum spanning tree algorithms in neural graph-based dependency parsers, indicating the need for alternative calibration techniques.

Contribution

The paper provides a theoretical proof that temperature scaling cannot alter the dependency parsing results derived from neural network scores.

Findings

Temperature scaling does not change maximum spanning tree outputs.

Calibration techniques must be different from temperature scaling for dependency parsers.

Miscalibration issues require new solutions beyond temperature scaling.

Abstract

Modern graph-based syntactic dependency parsers operate by predicting, for each token within a sentence, a probability distribution over its possible syntactic heads (i.e., all other tokens) and then extracting a maximum spanning tree from the resulting log-probabilities. Nowadays, virtually all such parsers utilize deep neural networks and may thus be susceptible to miscalibration (in particular, overconfident predictions). In this paper, we prove that temperature scaling, a popular technique for post-hoc calibration of neural networks, cannot change the output of the aforementioned procedure. We conclude that other techniques are needed to tackle miscalibration in graph-based dependency parsers in a way that improves parsing accuracy.