SEGAR: Selective Enhancement for Generative Augmented Reality

TL;DR

SEGAR introduces a framework combining diffusion-based world models with selective correction to generate and refine augmented reality frames, enabling real-time, coherent AR experiences with safety-critical adjustments.

Contribution

This work presents a novel approach integrating generative diffusion models with selective correction for AR, supporting precomputed, region-specific augmented frames with real-time safety adjustments.

Findings

Supports temporally coherent augmented frames

Enables precomputing and caching of future AR scenes

Demonstrates effectiveness in driving scenarios

Abstract

Generative world models offer a compelling foundation for augmented-reality (AR) applications: by predicting future image sequences that incorporate deliberate visual edits, they enable temporally coherent, augmented future frames that can be computed ahead of time and cached, avoiding per-frame rendering from scratch in real time. In this work, we present SEGAR, a preliminary framework that combines a diffusion-based world model with a selective correction stage to support this vision. The world model generates augmented future frames with region-specific edits while preserving others, and the correction stage subsequently aligns safety-critical regions with real-world observations while preserving intended augmentations elsewhere. We demonstrate this pipeline in driving scenarios as a representative setting where semantic region structure is well defined and real-world feedback is readily available. We view this as an early step toward generative world models as practical AR infrastructure, where future frames can be generated, cached, and selectively corrected on demand.