Towards Realistic Example-based Modeling via 3D Gaussian Stitching
Xinyu Gao, Ziyi Yang, Bingchen Gong, Xiaoguang Han, Sipeng Yang,, Xiaogang Jin
TL;DR
This paper introduces a novel example-based 3D modeling method that combines multiple Gaussian fields with a user-friendly GUI, enabling realistic, seamless composition and texture blending of 3D models from real-world scenes.
Contribution
It proposes a new workflow using 3D Gaussian Splatting, sample-guided synthesis, and a GUI for real-time segmentation, transformation, and harmonized blending of 3D models.
Findings
Outperforms previous methods in realistic synthesis quality
Enables real-time editing and composition of 3D models
Preserves rich textures during blending
Abstract
Using parts of existing models to rebuild new models, commonly termed as example-based modeling, is a classical methodology in the realm of computer graphics. Previous works mostly focus on shape composition, making them very hard to use for realistic composition of 3D objects captured from real-world scenes. This leads to combining multiple NeRFs into a single 3D scene to achieve seamless appearance blending. However, the current SeamlessNeRF method struggles to achieve interactive editing and harmonious stitching for real-world scenes due to its gradient-based strategy and grid-based representation. To this end, we present an example-based modeling method that combines multiple Gaussian fields in a point-based representation using sample-guided synthesis. Specifically, as for composition, we create a GUI to segment and transform multiple fields in real time, easily obtaining a…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
The idea of using differential coordinates (as in Poisson image editing, or, put alternatively, editing in higher-frequency bands of the appearance signal only) in the context of editing "Gaussian Splatting" scenes is quite appealing; if it wasn't for other reservations, I would consider this a sufficient argument for accepting the paper (I should note, however, that I am not an active researcher in this area and thus might overlook prior work; but if this is new, I would think it is a really ni
First of all, I have the impression that this paper is out of scope of ICLR. In terms of its approach and methodology as well as the problem it addresses, I would see this firmly within "computer graphics and interactive techniques"; it has only tangential impact on machine learning and representation learning. It might be close enough to warrant consideration, but I would see a graphics venue as a far better fit (for example, in computer graphics people would be much more interested in how this
The paper is well-written, logically clear, and readable. The proposed method is reasonable and improves the Nerf-based method by seamlessly integrating 3DGS with the real-world model. The paper provides experimental results to support its claims. Visualization results demonstrate the effectiveness of the method.
1. In Figure 13, the color propagation results are not clear. Could the authors provide examples with stronger color contrast or offer a detailed explanation of the color propagation, especially how the method handles various texture complexities? 2. The appendix briefly mentions time consumption comparisons, but could the authors elaborate on the computational demands of their method in the main text, perhaps comparing it to other existing methods in more detailed scenarios? 3. The paper menti
1. This paper proposes an effective method for composing pretrained 3D Gaussians, ensuring geometric accuracy and successfully addressing stylization between different objects. 2. A novel sampling-based optimization strategy is introduced to maintain the consistency of texture color between two 3D Gaussians, demonstrating the authors’ deep insight into object composition. 3. Authors develop a user-friendly GUI for composition and editing, which seems like a useful technical contribution for th
1. Please use the same cases as those in seamless NeRF (e.g., Figures 4, 5, 6, and 7 from seamless NeRF) to ensure a fair comparison. 2. Providing only VQA scores and images/videos is not sufficient for a convincing evaluation. I recommend that the authors introduce a more robust evaluation metric, rather than relying solely on videos/results. 3. A user-friendly GUI is primarily a technical contribution rather than a theoretical one, even though it is emphasized in Section 1. While there is te
* The proposed approach addresses a key gap in example-based modeling by enabling realistic and seamless stitching of 3D objects from real-world scenes, which previous techniques like SeamlessNeRF struggle with. This makes the method highly applicable for real-world applications that require detailed, cohesive compositions. * Extensive experimentation shows that this approach achieves high-quality, harmonious blends even in challenging real-world cases, where traditional neural field blending me
* Although the paper’s experiments on real-world data show improvements over SeamlessNeRF, it could further benefit from a more diverse dataset, as the current scope primarily includes simple object compositions without significant lighting variation or occlusion. Expanding the dataset to more complex scenes or settings with varied lighting conditions and object types could better showcase the generalizability of the approach. * The paper primarily demonstrates its effectiveness on objects with
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Topics3D Shape Modeling and Analysis · Image Processing and 3D Reconstruction · Robotics and Sensor-Based Localization
MethodsFocus