Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation

TL;DR

This paper introduces Bezier Trajectory Matching, a structured surrogate approach for dataset condensation that improves model training efficiency and performance on clinical datasets by better aligning supervision signals.

Contribution

It provides a geometric analysis of trajectory matching and proposes Bezier surrogates to enhance supervision structure and reduce storage in clinical dataset condensation.

Findings

BTM outperforms standard trajectory matching in clinical datasets.

BTM is especially effective in low-prevalence and low-budget scenarios.

Structured supervision signals improve dataset condensation efficiency.

Abstract

Dataset condensation constructs compact synthetic datasets that retain the training utility of large real-world datasets, enabling efficient model development and potentially supporting downstream research in governed domains such as healthcare. Trajectory matching (TM) is a widely used condensation approach that supervises synthetic data using changes in model parameters observed during training on real data, yet the structure of this supervision signal remains poorly understood. In this paper, we provide a geometric characterisation of trajectory matching, showing that a fixed synthetic dataset can only reproduce a limited span of such training-induced parameter changes. When the resulting supervision signal is spectrally broad, this creates a conditional representability bottleneck. Motivated by this mismatch, we propose Bezier Trajectory Matching (BTM), which replaces SGD trajectories with quadratic Bezier trajectory surrogates between initial and final model states. These surrogates are optimised to reduce average loss along the path while replacing broad SGD-derived supervision with a more structured, lower-rank signal that is better aligned with the optimisation constraints of a fixed synthetic dataset, and they substantially reduce trajectory storage. Experiments on five clinical datasets demonstrate that BTM consistently matches or improves upon standard trajectory matching, with the largest gains in low-prevalence and low-synthetic-budget settings. These results indicate that effective trajectory matching depends on structuring the supervision signal rather than reproducing stochastic optimisation paths.