Materialistic: Selecting Similar Materials in Images
Prafull Sharma, Julien Philip, Micha\"el Gharbi, William T. Freeman,, Fredo Durand, Valentin Deschaintre

TL;DR
Materialistic introduces a robust, similarity-based method for selecting regions of images with the same material, regardless of shading or lighting, using unsupervised features and a new synthetic dataset.
Contribution
The paper proposes a novel approach leveraging unsupervised DINO features and a Cross-Similarity module for material-based image region selection, trained on a synthetic dataset and applicable to real images.
Findings
Method generalizes well to real-world images
Effective in material editing and retrieval applications
Robust to shading, highlights, and shadows
Abstract
Separating an image into meaningful underlying components is a crucial first step for both editing and understanding images. We present a method capable of selecting the regions of a photograph exhibiting the same material as an artist-chosen area. Our proposed approach is robust to shading, specular highlights, and cast shadows, enabling selection in real images. As we do not rely on semantic segmentation (different woods or metal should not be selected together), we formulate the problem as a similarity-based grouping problem based on a user-provided image location. In particular, we propose to leverage the unsupervised DINO features coupled with a proposed Cross-Similarity module and an MLP head to extract material similarities in an image. We train our model on a new synthetic image dataset, that we release. We show that our method generalizes well to real-world images. We carefully…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Retrieval and Classification Techniques · Advanced Image and Video Retrieval Techniques · Visual Attention and Saliency Detection
MethodsMulti-Head Attention · Attention Is All You Need · Softmax · Layer Normalization · Linear Layer · Dense Connections · Residual Connection · Vision Transformer
