Memory-Guided View Refinement for Dynamic Human-in-the-loop EQA

TL;DR

This paper introduces a new framework called DIVRR for dynamic embodied question answering that improves robustness to occlusions and view changes by refining views and selectively managing memory, enabling efficient and accurate reasoning in dynamic scenes.

Contribution

The paper proposes DIVRR, a training-free, relevance-guided view refinement and memory selection framework for dynamic EQA, addressing occlusion ambiguity and evidence management.

Findings

DIVRR enhances robustness in dynamic scenes with occlusions.

DIVRR maintains high inference efficiency with compact memory.

Experiments show consistent improvements over baselines on DynHiL-EQA and HM-EQA datasets.

Abstract

Embodied Question Answering (EQA) has traditionally been evaluated in temporally stable environments where visual evidence can be accumulated reliably. However, in dynamic, human-populated scenes, human activities and occlusions introduce significant perceptual non-stationarity: task-relevant cues are transient and view-dependent, while a store-then-retrieve strategy over-accumulates redundant evidence and increases inference cost. This setting exposes two practical challenges for EQA agents: resolving ambiguity caused by viewpoint-dependent occlusions, and maintaining compact yet up-to-date evidence for efficient inference. To enable systematic study of this setting, we introduce DynHiL-EQA, a human-in-the-loop EQA dataset with two subsets: a Dynamic subset featuring human activities and temporal changes, and a Static subset with temporally stable observations. To address the above challenges, we present DIVRR (Dynamic-Informed View Refinement and Relevance-guided Adaptive Memory Selection), a training-free framework that couples relevance-guided view refinement with selective memory admission. By verifying ambiguous observations before committing them and retaining only informative evidence, DIVRR improves robustness under occlusions while preserving fast inference with compact memory. Extensive experiments on DynHiL-EQA and the established HM-EQA dataset demonstrate that DIVRR consistently improves over existing baselines in both dynamic and static settings while maintaining high inference efficiency.