MultiGrain: a unified image embedding for classes and instances
Maxim Berman, Herv\'e J\'egou, Andrea Vedaldi, Iasonas Kokkinos,, Matthijs Douze

TL;DR
MultiGrain introduces a unified network architecture that produces versatile image embeddings suitable for classification and object retrieval, achieving state-of-the-art accuracy without additional labels.
Contribution
It presents a novel embedding architecture with a pooling layer that captures coarse and fine details, improving both classification and retrieval performance.
Findings
Achieves 79.4% top-1 accuracy on ImageNet with ResNet-50.
Embeddings perform on par with state-of-the-art retrieval methods.
Simple joint training with classification and ranking losses enhances versatility.
Abstract
MultiGrain is a network architecture producing compact vector representations that are suited both for image classification and particular object retrieval. It builds on a standard classification trunk. The top of the network produces an embedding containing coarse and fine-grained information, so that images can be recognized based on the object class, particular object, or if they are distorted copies. Our joint training is simple: we minimize a cross-entropy loss for classification and a ranking loss that determines if two images are identical up to data augmentation, with no need for additional labels. A key component of MultiGrain is a pooling layer that takes advantage of high-resolution images with a network trained at a lower resolution. When fed to a linear classifier, the learned embeddings provide state-of-the-art classification accuracy. For instance, we obtain 79.4% top-1…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Advanced Neural Network Applications · Digital Imaging for Blood Diseases
MethodsSigmoid Activation · Tanh Activation · Average Pooling · PCA Whitening · Generalized Mean Pooling · Residual Connection · 1x1 Convolution · MultiGrain · Cutout · Color Jitter
