SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Grounding

TL;DR

SlotVTG enhances video temporal grounding by integrating object-centric reasoning into multimodal models, significantly improving out-of-domain generalization with minimal additional training.

Contribution

It introduces a lightweight slot adapter that promotes object-centric visual reasoning in MLLMs, improving OOD robustness without extensive retraining.

Findings

Significant improvement in OOD generalization on VTG benchmarks.

Maintains competitive In-Domain performance.

Minimal computational overhead compared to full retraining.

Abstract

Multimodal Large Language Models (MLLMs) have shown strong performance on Video Temporal Grounding (VTG). However, their coarse recognition capabilities are insufficient for fine-grained temporal understanding, making task-specific fine-tuning indispensable. This fine-tuning causes models to memorize dataset-specific shortcuts rather than faithfully grounding in the actual visual content, leading to poor Out-of-Domain (OOD) generalization. Object-centric learning offers a promising remedy by decomposing scenes into entity-level representations, but existing approaches require re-running the entire multi-stage training pipeline from scratch. We propose SlotVTG, a framework that steers MLLMs toward object-centric, input-grounded visual reasoning at minimal cost. SlotVTG introduces a lightweight slot adapter that decomposes visual tokens into abstract slots via slot attention and reconstructs the original sequence, where objectness priors from a self-supervised vision model encourage semantically coherent slot formation. Cross-domain evaluation on standard VTG benchmarks demonstrates that our approach significantly improves OOD robustness while maintaining competitive In-Domain (ID) performance with minimal overhead.