3D-LATTE: Latent Space 3D Editing from Textual Instructions

TL;DR

This paper introduces 3D-LATTE, a training-free method for high-fidelity, precise 3D asset editing directly in the latent space of a 3D diffusion model, overcoming view-inconsistency issues of prior approaches.

Contribution

It presents a novel latent space editing technique that combines attention map blending, spectral modulation, and regularization for improved 3D asset editing without additional training.

Findings

Outperforms previous 3D editing methods in fidelity and accuracy

Enables a wide range of semantic and geometric manipulations

Operates without training, reducing computational costs

Abstract

Despite the recent success of multi-view diffusion models for text/image-based 3D asset generation, instruction-based editing of 3D assets lacks surprisingly far behind the quality of generation models. The main reason is that recent approaches using 2D priors suffer from view-inconsistent editing signals. Going beyond 2D prior distillation methods and multi-view editing strategies, we propose a training-free editing method that operates within the latent space of a native 3D diffusion model, allowing us to directly manipulate 3D geometry. We guide the edit synthesis by blending 3D attention maps from the generation with the source object. Coupled with geometry-aware regularization guidance, a spectral modulation strategy in the Fourier domain and a refinement step for 3D enhancement, our method outperforms previous 3D editing methods enabling high-fidelity and precise edits across a wide range of shapes and semantic manipulations. Our project webpage is https://mparelli.github.io/3d-latte

Figures

Tables

Table 1. Table 1 : Quantitative comparison using CLIP score metrics . Best , second-best and third-best results are indicated.

Method	CLIP Dir ↑	CLIP Diff No-Edit ↓	CLIP-Dir-Con ↑
MVEdit [mvedit2024]	0.121	0.077	\cellcoloryellow!200.67
Vox-E [sella2023vox]	\cellcoloryellow!200.129	0.054	\cellcolororange!200.68
GaussCTRL [wu2024gaussctrl]	0.076	\cellcolorred!200.035	0.61
Edit360 [huang2025edit360]	\cellcolororange!200.149	\cellcoloryellow!200.045	0.59
PDS [Koo2024PDS]	0.051	0.094	0.55
InstructGS2GS [igs2gs]	0.069	0.082	0.64
3D-LATTE (Ours)	\cellcolorred!200.178	\cellcolororange!200.039	\cellcolorred!200.77

Table 2. Table 2 : Quantitative comparison using GPTEval3D. Following [ erkoc2024preditor3d ] , the scores represent the percentage of comparisons where our method wins over the respective baselines.

Method	Prompt Algn. ↑	3D Plausibility ↑	Texture ↑
MVEdit [mvedit2024]	87%	71%	70%
Vox-E [sella2023vox]	78%	81%	78%
GaussCTRL [wu2024gaussctrl]	94%	83%	81%
Edit360 [huang2025edit360]	67%	90%	72%

Equations

$$z_t=\sqrt{{\alpha }_t}{z}_0+\sqrt{1-{\alpha }_t}{ϵ}_t,{\eta }_t=\frac{(z_{t-1}-{\mu }_{\theta }(z_t,t))}{{\sigma }_t}$$

$${\widehat{W_{\mathcal{G}}}}_t^{\text{cross}}$$

$$=$$

$${\widehat{W_{\mathcal{G}}}}_t^{\text{self}}$$

$$=$$

$${\hat{z}}_{t-1}=(1-M)\odot z_{t-1}+M\odot z_{t-1}^{\ast }$$

$${\mathcal{L}}_{\text{geo}}={\lambda }_o\sum _i{R}_t^i\cdot \exp (-{\gamma }_o\cdot o_i)+{\lambda }_{\Sigma }\sum _i{R}_t^i\cdot \exp (-{\gamma }_{\Sigma }\cdot \mathrm{Tr}({\Sigma }_i))$$

$$z_{t-1}={\hat{z}}_{t-1}-s\cdot {\nabla }_{z_t}{\mathcal{L}}_{\text{geo}}(z_t)$$

$$\begin{array}{rlrl}& \begin{array}{rl}F(h_{l,t})& =\mathrm{FFT}(h_{l,t})\\ F^{\prime }(h_{l,t})& =F(h_{l,t})\odot {\beta }_{l,t}\\ h_{l,t}^{\prime }& =\mathrm{IFFT}(F^{\prime }(h_{l,t}))\end{array}& & \begin{array}{rl}{\beta }_{l,t}(r)& =\{\begin{array}{ll}{s}_l,& \text{if }t>\tau \text{ and }r<r_{\text{thresh}},\\ s_h,& \text{if }t\le \tau \text{ and }r\ge r_{\text{thresh}},\\ 1,& \text{otherwise.}\end{array}\end{array}\end{array}$$

Full text

1 Introduction

Advances in 2D diffusion models [rombach2021highresolution] and 3D representations, such as Neural Radiance Fields (NeRFs) [mildenhall2020nerf] and 3D Gaussian Splatting (3DGS) [kerbl3Dgaussians], have revolutionized 3D asset creation. More recently, powerful text-to-3D [lin2025diffsplat, xiang2024structured] and image-to-3D [lin2025diffsplat, xiang2024structured, liu2023zero1to3, liu2023one2345++] generative models have emerged, enabling scalable and expressive 3D content creation by learning from large 3D data collections [Deitke2023objaverse, objaverseXL]. A core challenge in this domain is instruction-based 3D editing: modifying the geometry and appearance of a 3D object based on a language instruction, while selectively preserving the object’s identity and structure. Achieving semantically accurate and high-fidelity 3D edits has attracted considerable attention due to its importance for applications in design and virtual and augmented reality.

Many methods propose to solve this task by distilling 2D diffusion priors [brooks2023instructpix2pix] into a 3D representation via score distillation losses [poole2022dreamfusion, li2023focaldreamer, sella2023vox, Koo2024PDS] or iterative dataset updates [Haque2023instructnerf, chen2023gaussianeditor, Chen2024proedit]. These approaches, however, are constrained by their reliance on 2D supervision. Specifically, they often exhibit multi-view inconsistencies, such as multi-face Janus issues, due to the limited 3D awareness, or inherit 2D editing failures from specific viewpoints, resulting in implausible 3D reconstructions. As a result, most existing methods succeed in appearance edits but struggle with large spatial or geometrical transformations that require globally consistent shape changes. A new line of work [erkoc2024preditor3d, li2025cmd, huang2025edit360] proposes to synchronously edit multiple views via multi-view diffusion priors or relies on feedforward 3D reconstruction models [zhuang2024gtr, LaRa, hong2023lrm] to consolidate them in 3D. Others adopt a hybrid 2D-3D approach [mvedit2024], where multi-view images are fused into a 3D representation at each denoising step. Nevertheless, existing methods based on feedforward 3D reconstruction or multi-view priors often suffer from blurry, distorted reconstructions, due to small inconsistencies across views being propagated to the 3D model. Hybrid 2D-3D methods, on the other hand, may introduce Janus artifacts and multi-view inconsistencies due to the use of 2D priors at intermediate steps.

To overcome these limitations, we take an alternative approach and leverage a 3D-native diffusion prior, where noise is injected directly to the 3D representation. This allows us to directly manipulate the appearance and geometry of a 3D asset without reliance on 2D or multi-view priors. Importantly, we operate within the model’s latent space, which is structured as pixel-aligned 3D Gaussian representations [lin2025diffsplat]. Motivated by the role of attention maps in 2D editing [hertz2022prompt], our key insight is that the 3D self- and cross-attention maps naturally capture information about the layout and composition of the 3D scene and directly model relationships between 3D Gaussians and text tokens. Building on this observation, we introduce a 3D attention injection mechanism into the denoising process of a text-conditioned 3D diffusion model. At each time step, we modulate the model’s attention maps by blending or replacing them with those obtained from the generation with the source prompt, namely the description of the original, unedited asset. This allows us to guide the model to synthesize a 3D asset that semantically aligns with the edit prompt while preserving the 3D structure of the source asset.

To achieve localized edits, we leverage a vision-language model (VLM) in combination with a segmentation model to generate multi-view consistent 2D masks. These masks naturally define a 3D segmentation over the multi-view pixel-aligned 3D Gaussians, allowing us to constrain the modification to the relevant 3D regions. To enhance 3D quality, we adopt a frequency-modulated strategy that emphasizes low-frequency components early in the denoising process, encouraging the model to capture global structure before refining fine-grained details. Structural coherence is further reinforced through a geometry-aware regularization term, applied in the form of classifier guidance. Finally, to address higher-fidelity edits we adopt an iterative strategy that progressively enhances the fine-grained details of the 3D representation while preserving cross-view consistency.

To demonstrate the effectiveness of our approach, we conduct extensive user studies, report quantitative metrics based on CLIP similarity [Haque2023instructnerf] and evaluate performance using GPTEval3D [wu2023gpteval3d], consistently surpassing previous state-of-the-art works.

2 Related Works

3D Diffusion Generative Models. Recent methods in 3D generation have shifted their focus toward diffusion-based models [ho2020denoising], which have been adapted to a plethora of 3D representations, including voxel grids [hui2022neural, trellis], point clouds [zeng2022lion, nichol2022pointe] and triplanes [Shue2023TriDiff, anciukevicius2022renderdiffusion]. Recently, methods leveraging 3DGS [yang2024prometheus, Zhou2025diffgs, lin2025diffsplat, zhang2024gaussiancube] have emerged as particularly promising. GaussianCube [zhang2024gaussiancube] organizes a 3DGS representation into a voxel field and applies 3D diffusion using a 3D U-Net. In contrast, DiffSplat [lin2025diffsplat] introduces a 3D latent diffusion model that directly generates 3D Gaussians by modeling an object as a set of multi-view 3DGS grids.

3D Editing with 2D Priors. Editing in 3D presents unique challenges due to the need for multi-view consistency. One line of work tackles this by leveraging image-conditioned 2D diffusion models such as InstructPix2Pix [brooks2023instructpix2pix], injecting their editing capabilities into learned 3D representations. A pioneering work in this direction, InstructNeRF2NeRF [Haque2023instructnerf], introduces the “Iterative Dataset Update” method, where rendered views from the 3D model are repeatedly updated during optimization. More recent methods [chen2023gaussianeditor, Chen2024proedit], build upon this paradigm. However, large edits can still lead to multi-view inconsistencies due to the lack of explicit multi-view awareness. Another line of work,  [zhuang2023dreameditor, li2023focaldreamer, sella2023vox], relies on Score Distillation Sampling (SDS) to guide 3D asset synthesis by distilling gradients from a pre-trained diffusion model. Posterior Distillation Sampling [Koo2024PDS] improves upon SDS by aligning the latents of source and target images. While effective, these methods inherit known limitations of SDS, including Janus artifacts and mode-seeking behavior. As an alternative, recent methods [wu2024gaussctrl, lee2025editsplat, chen2024dge, wen2025intergsedit] perform multi-view consistent edits directly on source images, which are then used to update the 3D representation. For instance, DGE [chen2024dge] leverages epipolar constraints to aggregate features across views, while GaussCTRL [wu2024gaussctrl] performs depth-conditioned 2D updates combined with cross-view alignment. However, such reliance on depth guidance can prevent the model from performing significant shape changes. In contrast, by operating directly within the latent space of a 3D diffusion model, our method enables flexible manipulation of both appearance and geometry in a fully 3D-consistent manner without relying on 2D editing priors or SDS-based optimization.

Hybrid 2D-3D Editing Methods. In parallel, another line of work has emerged that moves beyond pure 2D updates by incorporating hybrid 2D-3D strategies. SHAP-Editor [chen2024shap] learns a feed-forward editor operating in the latent space of Shap-E [Jun2023ShapEGC] and is trained using an SDS [poole2022dreamfusion] objective. However, it requires retraining for each new set of edits and relies heavily on the 2D priors and latent structure of Shap-E [Jun2023ShapEGC]. Thus, its edits lack flexibility and are often of poor visual quality. In contrast, our method generalizes across categories, produces high-fidelity outputs, and enables fast inference. MVEdit [mvedit2024] adopts a hybrid 2D-3D approach by fusing images into a 3D representation between denoising steps in a multi-view diffusion model. More recently, Edit360 [huang2025edit360] builds on a dense-view synthesis model and achieves propagation of edits by aligning an edited view camera trajectory with the original front-view trajectory. While these are promising steps toward enabling 3D-aware edits, reliance on 2D priors or trajectory alignment strategies can introduce multi-view inconsistencies.

3 Method

Given a 3D object, a source text prompt $p$ describing its original appearance and a target text prompt $p^{\ast}$ describing the desired edit, our goal is to alter the object’s appearance and/or geometry so that it semantically aligns with the target prompt. Simultaneously, we aim to preserve regions not referenced by the prompt and maintain the original object structure. To this end, we introduce a zero-shot editing framework that extends attention control to the domain of 3DGS. Our method operates within the text-guided 3D diffusion model DiffSplat [lin2025diffsplat], outlined in Section 3.1. Our core idea is to inject 3D cross- and self-attention maps derived from the source 3D asset during the diffusion process that synthesizes the edited 3D asset as explained in Section 3.3. This enables semantically meaningful edits while maintaining multi-view consistency and 3D structural coherence, since our attention modifications are applied within a latent space that encodes 3D geometry. To further enhance quality, we incorporate a geometry-aware regularization mechanism (Section 3.5) and a frequency annealing strategy (Section 3.6). Finally, in Section 3.7 we introduce an iterative refinement pipeline that progressively enhances high-frequency details and texture fidelity in the reconstructed 3D asset. An overview of our method is shown in Figure 2.

3.1 3D Representation and 3D Diffusion Model

In this work, we leverage DiffSplat [lin2025diffsplat] as our 3D diffusion backbone. In [lin2025diffsplat] a 3D object is modeled as a set of structured multi-view splat grids $\mathcal{G}=\{G_{i}\}_{i=1}^{V}$, where each $G_{i}\in\mathbb{R}^{C\times H\times W}$, $V$ is the number of input views, $H\times W$ is the spatial resolution, and $C=12$ corresponds to the number of Gaussian attributes. Each Gaussian primitive $g_{i}\in\mathbb{R}^{12}$ is parameterized by its RGB color $c_{i}\in\mathbb{R}^{3}$, 3D location $x_{i}\in\mathbb{R}^{3}$, which is determined by its depth and camera parameters, scale $s_{i}\in\mathbb{R}^{3}$, rotation quaternion $r_{i}\in\mathbb{R}^{4}$, and opacity $o_{i}\in\mathbb{R}$. The pipeline of [lin2025diffsplat] comprises a Gaussian reconstruction module that converts multi-view RGB images, along with depth and normal maps into a Gaussian splat grid representation, a VAE that encodes the Gaussian splat grid into a latent space and a generative model that is trained on this representation. Finally, the reconstructed latent is fed to a VAE decoder to obtain the 3D gaussian representation, which can be rendered as multi-view images.

3.2 3D Inversion

Following [lin2025diffsplat], we represent a 3D object as a set of structured multi-view splat grids $\mathcal{G}=\{G_{i}\}_{i=1}^{V}$. We use the reconstruction module from DiffSplat [lin2025diffsplat], which takes as input a set of $V$ RGB images, depth maps, and normal maps rendered from uniformly distributed camera viewpoints around the source 3D object, resulting in a 3D Gaussian for each pixel of the input views. We begin by inverting the source 3D object representation into its corresponding noise latent $z_{T}$. However, standard DDIM inversion [song2021ddim] can introduce slight errors, leading to poor reconstructions. Thus, we adapt the DDPM inversion mechanism proposed in  [huberman2024edit] to operate on a set of multi-view Gaussian splat grids $\mathcal{G}$ and recover a noise vector that accurately reconstructs the original 3D object.

Let the encoded Gaussian splat grids (i.e., splat latents) be denoted as $z_{0}=\{z_{0}^{i}\}_{i=1}^{V}=\{E_{\phi}(G_{i})\}_{i=1}^{V}$ where ${E_{\phi}}(\cdot)$ is the Gaussian VAE encoder of [lin2025diffsplat]. We construct the forward diffusion trajectory [song2021ddim] and obtain the noise maps $\eta_{t}$ as follows:

[TABLE]

where $\alpha_{t}$ is the cumulative diffusion noise schedule, $\epsilon_{t}\sim\mathcal{N}(0,I)$ is sampled independently per timestep, and $\mu_{\theta}$ and $\sigma_{t}$ are the predicted mean and standard deviation of the noise injected at each timestep, respectively. This sequence of noise maps is shown in [huberman2024edit] to be edit-friendly since they are constructed with statistically independent noise samples. Starting from $z_{T}$, we perform the denoising process with the edit prompt $p^{\ast}$. To achieve editing while maintaining 3D structural information and layout, we apply 3D self- and cross-attention injection.

3.3 3D Attention Injection

In our formulation, 3D-aware attention refers to the self- and cross-attention maps over the noisy multi-view Gaussian latents. In our 3D diffusion backbone at each timestep, features of the noisy Gaussian splat latents $\phi({z}_{t})\in\mathbb{R}^{V\times D\times H\times W}$ are projected into a query matrix, $Q$, while the keys, $K$, and values, $V$, are obtained from the text prompt embeddings. Each entry $W_{i,j}$ in the 3D cross-attention map $W_{\mathcal{G}}^{\text{cross}}$ represents the influence of the $j$-th text token on the $i$-th Gaussian latent. As illustrated in Figure 3, cross-attention weights define a token-3D Gaussian splat correspondence field, enabling accurate 3D localization of interest regions. Self-attention maps $W_{\mathcal{G}}^{\text{self}}$ capture spatial and semantic relationships between all 3D Gaussian latents.

Queries, keys and values are computed via linear projections of the Gaussian features.

Motivated by the role of attention in encoding 3D structure and semantics within the 3D Gaussian splat representation, we introduce a 3D attention injection framework. Let $z_{t-1}=D_{\theta}(z_{t},t,p)$ denote the denoising step with the source prompt $p$, and $z^{\ast}_{t-1}=D_{\theta}(z_{t}^{*},t,p^{*})$ be the denoising step with the edit prompt $p^{*}$. Additionally, let ${W_{\mathcal{G}}}_{t}=({W_{\mathcal{G}}}^{\text{cross}}_{t},{W_{\mathcal{G}}}_{t}^{\text{self}})$ and ${W_{\mathcal{G}}}_{t}^{\ast}=(({W_{\mathcal{G}}}_{t}^{\ast})^{\text{cross}},({W_{\mathcal{G}}}_{t}^{\ast})^{\text{self}})$ be the attention maps computed in the denoising steps $D_{\theta}(z_{t},t,p)$ and $D_{\theta}(z_{t}^{*},t,p^{*})$, respectively. We inject modified attention maps $\hat{W_{\mathcal{G}}}_{t}=(\hat{W_{\mathcal{G}}}_{t}^{\text{cross}},\hat{W_{\mathcal{G}}}_{t}^{\text{self}})$, derived from the source prompt $p$, into the denoising process guided by the edit prompt $p^{\ast}$, as follows. At each denoising step with prompt $p^{\ast}$, we override the attention maps ${W_{\mathcal{G}}}^{\ast}_{t}$ with $\hat{W_{\mathcal{G}}}_{t}$ (i.e., ${W_{\mathcal{G}}}^{\ast}_{t}\leftarrow\hat{W_{\mathcal{G}}}_{t}$). To calculate the modified cross-attention map, we follow [hertz2022prompt] and inject 3D attention until a timestep $\tau_{\text{cross}}$ only on tokens shared between the original and edited prompts, using the following $F_{cross}^{3D}$ function:

[TABLE]

where $CT$ is an alignment function that maps a token index from $p^{\ast}$ to the corresponding token index in $p$. In the case where there is no match, it returns an empty set.

By also injecting 3D self-attention during the early timesteps of the denoising process, we encourage the model to further preserve 3D structural relationships of the source asset, such as part layout, composition and spatial symmetry, before the introduction of semantic details. As illustrated in Figure 4, the spectral decomposition of the 3D self-attention graph reflects 3D scene composition and groups 3D Gaussians into distinct semantic parts. In the case of self-attention, the $F_{self}^{3D}$ function is defined as follows:

[TABLE]

where $\tau_{\text{self}}$ is the timestep until which injection is applied.

3.4 Mask Generation and Region Editing

To extract the target edit area, we formulate a query to a large vision-language model (GPT-4o) that includes the original prompt, the edit instruction, and a rendered front-view of the object. The model is asked to identify which part of the object is affected by the edit (e.g., “the shirt of the teddy bear”). We use these identified editing regions to generate 2D multi-view consistent segmentation masks. More specifically, GroundingDINO [liu2023grounding] processes the multi-view rendered images and the edit regions, proposed by GPT-4o to generate bounding boxes, which are subsequently refined into 2D segmentation masks tracked by SAM 2 [ravi2024sam2]. This process is outlined in the supplementary. Benefiting from our pixel-aligned representation, these masks naturally approximate a 3D segmentation of the corresponding pixel-aligned Gaussians. To allow for flexible geometric modifications, we approximate the initial 3D mask expansion by computing the cross-attention map $\overline{{W_{\mathcal{G}}}_{t}^{r^{\ast}}}$ averaged over all time steps for the prompt token $r^{\ast}$ that refers to the desired editing region (e.g., “tutu” in Figure 2), and applying a threshold to obtain binary masks that indicate 3D regions influenced by the new concept. The final 3D editing region $M$ is defined as the union of the original masks predicted by SAM 2 lifted to 3D and the attention-derived mask. We constrain the diffusion process to this area, by blending the original latent $z_{t-1}$ with the edited latent $z_{t-1}^{*}$ as follows:

[TABLE]

where $\odot$ denotes element-wise multiplication. Our method also supports user-defined masks.

3.5 Geometry Regularization

The editing signal via attention injection can introduce some uncertainty in the editing regions, resulting in Gaussians prone to semi-transparencies and premature collapse. We find that a simple geometric regularizer can help circumvent these artifacts. Thus, we incorporate a soft geometry-aware classifier guidance mechanism that softly penalizes the removal of Gaussians in regions that are highly relevant to the edit. To this end, we compute a soft mask $R_{t}^{i}\in[0,1]$ for each Gaussian $i$, which reflects how relevant that Gaussian is to the current edit at denoising step $t$. We define the editing signal at each timestep as the L1 difference between the noise predictions conditioned on the edited and original prompts: $D^{i}=\left\|\epsilon_{\theta}(z_{t},t,p^{\ast})-\epsilon_{\theta}(z_{t},t,p)\right\|_{1},$ where $\epsilon_{\theta}(z_{t},t,p)$ and $\epsilon_{\theta}(z_{t},t,p^{\ast})$ denote the denoising model’s noise predictions under the original and edited prompts, respectively. To obtain a soft relevance mask $R_{t}$ we normalize the absolute difference values $D^{i}$ across all Gaussians via global min–max normalization. Thus, $R_{t}^{i}$ denotes the relevance weight of the $i$-th Gaussian, arranged in a pixel-aligned grid of size $V\times H\times W$. Our intuition is that the magnitude of this mismatch reflects the relevance of each 3D Gaussian to the edit. As the existence of Gaussians is determined by their opacity and covariance, our regularization loss is defined as:

[TABLE]

where $o$ and $\Sigma$ are obtained by decoding the denoised gaussian splat latents with the VAE decoder $D_{\phi}$ from [lin2025diffsplat]. Here, $\gamma_{o}$ and $\gamma_{\Sigma}$ control the sharpness of the penalties, and $\lambda_{o}$, $\lambda_{\Sigma}$ weigh the opacity and scale terms. This formulation softly penalizes low opacity and insufficient spatial support. We incorporate our geometry-aware regularization loss as a guidance signal during denoising:

[TABLE]

where $s$ denotes the guidance scale, and $\hat{z}_{t-1}$ is the predicted latent from Equation 4.

3.6 Frequency Annealing

The injection of attention from the source prompt during editing can impact the model’s denoising ability, intensifying the misalignment between the condition and diffusion spaces. In some examples, this causes the model to overemphasize high-frequency visual features of the source, such as decorative textures. These may become partially preserved and degrade into artifacts on the surface of the generated 3D asset. Consistent with observations in 2D [wu2024freeinit, si2023freeu], in 3D generation, high-frequency components correspond to fine details while low-frequencies primarily capture 3D geometric structure and semantic layout. Thus, we introduce a frequency annealing strategy, where we apply spectral modulation in the Fourier domain of the feature maps from the U-Net skip connections at each denoising step. Given the gaussian feature map $h_{l,t}\in\mathbb{R}^{V\times C\times H\times W}$ at layer $l$ and timestep $t$, we apply the following steps:

[TABLE]

Here, $\text{FFT}(\cdot)$ and $\text{IFFT}(\cdot)$ denote the Fourier transform and its inverse. The modulation mask $\beta_{l,t}$ applies scaling based on the radius $r$ and the terms $s_{l}$ and $s_{h}$ denote the scaling factors applied to the low- and high-frequency components respectively. This encourages the model to boost low-frequency components during early denoising with attention injection and then transition to emphasize high-frequency components, enabling structural preservation of the source without unwanted oversmoothed textures.

3.7 3D Enhancement

Many existing text-to-3D generation models, particularly those fine-tuned on datasets with plain surface details, are constrained to lower-resolution outputs and struggle to capture fine-grained geometric and appearance details. Thus, in challenging cases involving finer structures when rendering our 3D Gaussian representation at higher resolutions, we observe a degradation in appearance quality, characterized by blurred textures and limited details. Drawing inspiration from [Haque2023instructnerf] to improve visual fidelity while preserving 3D structure and consistency, we leverage an iterative dataset update technique. Unlike prior work that primarily uses such pipelines for editing [Haque2023instructnerf, chen2023gaussianeditor], we adapt them for 3D enhancement. Specifically, we follow an iterative process that comprises three key steps: i) rendering high-resolution views from the edited 3DGS representation, ii) enhancing these views by inputting noised version of them to the 2D diffusion backbone model, conditioned on the original edited images and, (iii) re-optimizing the 3DGS model with the updated enhanced images. To constrain the extent of correction to the editing area, the updated image is represented as: $I_{\text{blend}}=M\odot I_{e}+(1-M)\odot I_{\text{src}},$ where $M$ denotes the mask containing the changed region, $I_{e}$ denotes the enhanced image and $I_{\text{src}}$ denotes the image of the source object. This process is shown in the supplementary.

As our 2D enhancement backbone, we leverage the ControlNet-Tile model [zhang2023adding], which is designed for structure-preserving super-resolution, and is effective at restoring high-frequency details. By progressively re-rendering and updating the images we converge to a globally consistent higher-fidelity 3D representation.

4 Experiments

To evaluate our method, we construct a benchmark of 25 diverse 3D assets, each paired with a set of distinct edit instructions, resulting in a total of 100 samples. The assets are sourced from the Objaverse [Deitke2023objaverse] and Google Scanned Objects (GSO) [Downs2022GSO] datasets. As main baselines, we include Vox-E [sella2023vox], a voxel-based method using SDS, MVEdit [mvedit2024], GaussCTRL [wu2024gaussctrl] and Edit360 [huang2025edit360]. For completeness, we also report quantitative results on two additional well-studied baselines: InstructGS2GS [igs2gs] and Posterior Distillation Sampling (PDS) [Koo2024PDS]. For the quantitative evaluation, we adopt the CLIP directional similarity (CLIP-dir) metric [Haque2023instructnerf] to quantify semantic alignment to edits by measuring direction changes in CLIP text embeddings and corresponding image embeddings of multi-view renderings. To measure how well unedited regions are preserved, we include the CLIP-diff-no-edit metric [erkoc2024preditor3d], which measures the CLIP score difference between input and output images using a modified text prompt where the edited part is replaced with a generic placeholder. Lower values indicate better shape preservation. Finally, to quantitatively evaluate if the edit propagates consistently across viewpoints we adopt the CLIP direction consistency score (CLIP-Dir-Con) [Haque2023instructnerf], which computes how much the editing directions differ across frames. More details about the metrics are presented in the supplementary. To complement these metrics, we report results from GPTEval3D [wu2023gpteval3d], a GPT-4V-based evaluation protocol in which the model compares multi-view renderings along three criteria: Text Prompt–3D Alignment, 3D Plausibility, and Texture Details.

4.1 Qualitative Results

As illustrated in Figure 6 and 5, our method produces high-quality, multi-view consistent edits that are faithful to the input prompt. It is capable of handling significant geometric transformations, such as turning a shovel into a flower, or a house into an ancient building while preserving unedited regions and the structural integrity of the original object. As shown in Figure 5, MVEdit [mvedit2024] successfully modifies appearance but struggles with morphological changes and suffers from multi-view inconsistencies. GaussCTRL [wu2024gaussctrl] exhibits similar limitations as it also struggles with geometric transformations, due to its reliance on depth-guidance. Vox-E [sella2023vox] handles geometric changes more robustly but suffers from low quality and sometimes fails to localize edits (e.g., wings on the character). Moreover, it is prone to unnatural 3D geometries due to the limited 3D awareness of SDS. Edit360 [huang2025edit360] shows promise in edit faithfulness, however, the fusion of front and edited view trajectories introduces significant multi-view inconsistencies. Additional comparisons can be found in the supplementary.

4.2 Quantitative Results

The quantitative comparison in Table 1 supports the qualitative findings that our method surpasses the baselines in terms of semantic alignment of predictions to the edit prompts measured by the CLIP-dir score. The CLIP-no-diff score demonstrates that we preserve shapes effectively, compared to the other methods and we achieve a balanced tradeoff between editing and shape preservation. While GaussCTRL achieves a lower Diff-No-Edit score, this can be attributed to its frequent failure to apply the edits, leaving the objects unaltered. This is reflected in its significantly lower CLIP-dir score and in the qualitative examples. In Table 2, we show the results of the GPTEval3D [wu2023gpteval3d] metric in which GPT-4V shows a distinct preference for our method.

We also conduct a user study with 57 participants, who were asked to choose the best editing results among ours and three baselines, based on two criteria: visual quality and faithfulness to the edit prompt. The results in Figure 7 show that our method was preferred by a significant margin.

4.3 Ablation study

We illustrate visual results showing the impact of the core components of our pipeline. Additional quantitative results are included in the supplementary material.

Effect of 3D Enhancement. Figure 8 showcases the effect of our proposed 3D enhancement module. The top/bottom rows show a rendered view of the 3D objects before and after enhancement, demonstrating that the enhancement module recovers fine details and sharpens textures while maintaining 3D geometry(e.g., sharper architectural details).

Effect of Geometry Regularization. We also qualitatively demonstrate the effectiveness of our geometry regularization guidance term. As shown in Figure 9(a), without this term, some edited regions become partially transparent or vanish entirely, leading to degraded geometry. Our regularization mitigates these issues, resulting in robust edits.

Effect of Frequency Annealing. As shown in Figure 9(b), without frequency annealing, complex patterns in the source object, such as logos or prints that contain high-frequency information, can be overemphasized by the model, resulting in noisy textures in the final edit.

5 Conclusion

In this paper, we introduced 3D-LATTE, a novel approach for instruction-based 3D editing. In contrast to existing approaches that rely on 2D distillation or multi-view priors, we leverage the expressiveness of the latent space of a native 3D diffusion model and perform editing by blending the respective 3D attention maps of the source and target object. Combined with geometry regularization, frequency annealing, and a 3D refinement step, our approach outperforms competing methods in both qualitative and quantitative evaluations and user studies.