AeroBridge-TTA: Test-Time Adaptive Language-Conditioned Control for UAVs

TL;DR

AeroBridge-TTA introduces a test-time adaptive control pipeline for language-guided UAVs, significantly improving out-of-distribution performance by online latent adaptation.

Contribution

It presents a novel test-time adaptation method that updates a learned latent online, enhancing UAV control under dynamic and mismatched conditions.

Findings

AeroBridge-TTA matches in-distribution performance and outperforms baselines out-of-distribution.

Achieves +22.0 points on average in OOD conditions, with a 4.6× OOD lift from latent updates.

Outperforms PPO-MLP baseline in 5 language-conditioned UAV tasks under 13 mismatch scenarios.

Abstract

Language-guided unmanned aerial vehicles (UAVs) often fail not from bad reasoning or perception, but from execution mismatch: the gap between a planned trajectory and the controller's ability to track it when the real dynamics differ from training (mass changes, drag shifts, actuator delay, wind). We propose AeroBridge-TTA, a language-conditioned control pipeline that targets this gap with test-time adaptation. It has three parts: a language encoder that maps the command into a subgoal, an adaptive policy conditioned on the subgoal and a learned latent, and a test-time adaptation (TTA) module that updates the latent online from observed transitions. On five language-conditioned UAV tasks under 13 mismatch conditions with the same domain randomization, AeroBridge-TTA ties a strong PPO-MLP baseline in-distribution and wins all 5 out-of-distribution (OOD) conditions, +22.0 pts on average (62.7% vs. 40.7%); the +8.5 pt overall gain comes entirely from the OOD regime. A same-weights ablation that only changes the step size $\alpha$ shows the latent update itself is responsible for a $4.6\times$ OOD lift.