Multiscale Memory Comparator Transformer for Few-Shot Video Segmentation
Mennatullah Siam, Rezaul Karim, He Zhao, Richard Wildes

TL;DR
This paper introduces a multiscale memory transformer for few-shot video segmentation that preserves detailed features across scales, leading to improved accuracy and state-of-the-art results.
Contribution
It proposes a novel multiscale memory comparator within a transformer decoder that maintains detailed features across scales, enhancing few-shot video segmentation performance.
Findings
Outperforms baseline methods on few-shot video object segmentation
Achieves state-of-the-art results across multiple tasks
Provides empirical insights into multiscale information exchange
Abstract
Few-shot video segmentation is the task of delineating a specific novel class in a query video using few labelled support images. Typical approaches compare support and query features while limiting comparisons to a single feature layer and thereby ignore potentially valuable information. We present a meta-learned Multiscale Memory Comparator (MMC) for few-shot video segmentation that combines information across scales within a transformer decoder. Typical multiscale transformer decoders for segmentation tasks learn a compressed representation, their queries, through information exchange across scales. Unlike previous work, we instead preserve the detailed feature maps during across scale information exchange via a multiscale memory transformer decoding to reduce confusion between the background and novel class. Integral to the approach, we investigate multiple forms of information…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques
