Stochastic-based Patch Filtering for Few-Shot Learning

TL;DR

This paper introduces SPFF, a stochastic patch filtering method that enhances few-shot food image classification by focusing on relevant patches, leading to improved accuracy on multiple benchmarks.

Contribution

The paper presents a novel stochastic patch filtering technique that selectively emphasizes class-relevant image regions in few-shot learning scenarios.

Findings

SPFF outperforms existing state-of-the-art methods on Food-101, VireoFood-172, and UECFood-256.

Qualitative analysis shows SPFF effectively filters non-relevant patches.

The method improves focus on class-specific features in complex food images.

Abstract

Food images present unique challenges for few-shot learning models due to their visual complexity and variability. For instance, a pasta dish might appear with various garnishes on different plates and in diverse lighting conditions and camera perspectives. This problem leads to losing focus on the most important elements when comparing the query with support images, resulting in misclassification. To address this issue, we propose Stochastic-based Patch Filtering for Few-Shot Learning (SPFF) to attend to the patch embeddings that show greater correlation with the class representation. The key concept of SPFF involves the stochastic filtering of patch embeddings, where patches less similar to the class-aware embedding are more likely to be discarded. With patch embedding filtered according to the probability of appearance, we use a similarity matrix that quantifies the relationship between the query image and its respective support images. Through a qualitative analysis, we demonstrate that SPFF effectively focuses on patches where class-specific food features are most prominent while successfully filtering out non-relevant patches. We validate our approach through extensive experiments on few-shot classification benchmarks: Food-101, VireoFood-172 and UECFood-256, outperforming the existing SoA methods.