Modernising Reinforcement Learning-Based Navigation for Embodied Semantic Scene Graph Generation

TL;DR

This paper improves embodied semantic scene graph generation by modernizing navigation decision-making, exploring discrete motion sets, and comparing policy architectures, leading to significant gains in scene graph completeness and safety.

Contribution

It introduces a modular navigation component with updated decision-making, including a factorized multi-head policy and new exploration strategies, enhancing scene graph quality and safety.

Findings

Replacing the optimization algorithm improves SSG completeness by 21%.

Depth-based collision supervision enhances safety without affecting completeness.

Finer-grained, factorized actions yield the best completeness-efficiency balance.

Abstract

Semantic world models enable embodied agents to reason about objects, relations, and spatial context beyond purely geometric representations. In Organic Computing, such models are a key enabler for objective-driven self-adaptation under uncertainty and resource constraints. The core challenge is to acquire observations maximising model quality and downstream usefulness within a limited action budget. Semantic scene graphs (SSGs) provide a structured and compact representation for this purpose. However, constructing them within a finite action horizon requires exploration strategies that trade off information gain against navigation cost and decide when additional actions yield diminishing returns. This work presents a modular navigation component for Embodied Semantic Scene Graph Generation and modernises its decision-making by replacing the policy-optimisation method and revisiting the discrete action formulation. We study compact and finer-grained, larger discrete motion sets and compare a single-head policy over atomic actions with a factorised multi-head policy over action components. We evaluate curriculum learning and optional depth-based collision supervision, and assess SSG completeness, execution safety, and navigation behaviour. Results show that replacing the optimisation algorithm alone improves SSG completeness by 21\% relative to the baseline under identical reward shaping. Depth mainly affects execution safety (collision-free motion), while completeness remains largely unchanged. Combining modern optimisation with a finer-grained, factorised action representation yields the strongest overall completeness--efficiency trade-off.