# ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning

## TL;DR

ReSURE is an adaptive training method that dynamically reduces the impact of unreliable supervision in multi-turn dialogue systems, improving stability and response quality across datasets of varying quality.

## Contribution

It introduces a novel online loss reweighting technique based on per-turn loss distributions, addressing turn-level error propagation without explicit data filtering.

## Key findings

- ReSURE improves dialogue response quality on mixed-quality datasets.
- It maintains positive correlation between response scores and sample count across benchmarks.
- The method enhances training stability and robustness in low-quality data scenarios.

## Abstract

Fine-tuning multi-turn dialogue systems requires high-quality supervision but often suffers from degraded performance when exposed to low-quality data. Supervision errors in early turns can propagate across subsequent turns, undermining coherence and response quality. Existing methods typically address data quality via static prefiltering, which decouples quality control from training and fails to mitigate turn-level error propagation. In this context, we propose ReSURE (Regularizing Supervision UnREliability), an adaptive learning method that dynamically down-weights unreliable supervision without explicit filtering. ReSURE estimates per-turn loss distributions using Welford's online statistics and reweights sample losses on the fly accordingly. Experiments on both single-source and mixed-quality datasets show improved stability and response quality. Notably, ReSURE enjoys positive Spearman correlations (0.21 ~ 1.0 across multiple benchmarks) between response scores and number of samples regardless of data quality, which potentially paves the way for utilizing large-scale data effectively. Code is publicly available at https://github.com/Elvin-Yiming-Du/ReSURE_Multi_Turn_Training.