TL;DR
Neural USD introduces a structured, hierarchical scene representation framework inspired by USD, enabling precise, iterative, and object-specific editing in generative models, addressing limitations of global scene changes.
Contribution
The paper presents Neural USD, a novel object-centric, hierarchical scene representation framework that allows for disentangled, per-object control in generative editing workflows.
Findings
Enables precise, iterative object editing without global scene alterations
Supports diverse signals and minimizes model constraints
Facilitates incremental scene editing workflows
Abstract
Amazing progress has been made in controllable generative modeling, especially over the last few years. However, some challenges remain. One of them is precise and iterative object editing. In many of the current methods, trying to edit the generated image (for example, changing the color of a particular object in the scene or changing the background while keeping other elements unchanged) by changing the conditioning signals often leads to unintended global changes in the scene. In this work, we take the first steps to address the above challenges. Taking inspiration from the Universal Scene Descriptor (USD) standard developed in the computer graphics community, we introduce the "Neural Universal Scene Descriptor" or Neural USD. In this framework, we represent scenes and objects in a structured, hierarchical manner. This accommodates diverse signals, minimizes model-specific…
Peer Reviews
Decision·Submitted to ICLR 2026
1. Achieves disentangled control over an object’s appearance and geometry in a simple and straightforward manner. 2. Enables more precise control of objects, with quantitative metrics outperforming previous methods. 3. Designs a stable iterative 3D editing workflow, allowing sequential replacement of pose, appearance, object, and background, while preserving the results of previous edits during new editing steps.
1. The paper repeatedly emphasizes that it can edit an object’s pose while keeping other attributes of the source image unchanged (as stated in the abstract, section 4.2). However, in practice, the camera pose and object pose are not successfully disentangled — pose editing often causes a change in the viewing angle instead of the object moving relative to the scene (see Fig. 9). 2. This work follows a technical route very similar to Neural Assets, with comparable metrics (see Figs. 7 and 8), ye
1. The idea is interesting to me, that is a unified, structured conditioning standard inspired by USD, enabling disentangled control over object appearance, geometry, and pose in generative models. 2. The ability to perform multi-step, fine-grained, object-level edits without unintended global changes, a clear improvement over existing conditioning methods (e.g., ControlNet, InstructPix2Pix). 3. The format is architecture-agnostic and supports diffusion, DiT, and transformer models through token
1. Global scene changes still occur, as I observed from the visual results. It is somewhat overclaimed, as the abstract and introduction sections stated. 2. It remains unclear whether the fusion happens at the feature level (joint embedding) or via concatenated conditioning channels. 3. Uses Stable Diffusion v2.1 as backbone, leading to lower image quality than state-of-the-art diffusion models; I still believe it should be adopted in more powerful backbones (e.g., Flux).
1. The paper targets a great problem for finer control of the objects in an image. The proposed method, although simple, is pretty straightforward and effective. 2. The datasets in the paper can benefit future research. 3. The demonstrated results are good.
1. Since the model is still a learning-based image-to-image model, keeping other objects unchanged is not guaranteed. I can see obvious background change in Figure 1.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
