SARE: Sample-wise Adaptive Reasoning for Training-free Fine-grained Visual Recognition

TL;DR

SARE is a training-free framework that adaptively combines retrieval and reasoning for fine-grained visual recognition, improving accuracy and efficiency by leveraging past failures without parameter updates.

Contribution

It introduces a cascaded, sample-wise adaptive reasoning approach with a self-reflective experience mechanism for training-free FGVR.

Findings

Achieves state-of-the-art performance on 14 datasets.

Reduces computational overhead compared to existing methods.

Effectively leverages past failures for improved inference.

Abstract

Recent advances in Large Vision-Language Models (LVLMs) have enabled training-free Fine-Grained Visual Recognition (FGVR). However, effectively exploiting LVLMs for FGVR remains challenging due to the inherent visual ambiguity of subordinate-level categories. Existing methods predominantly adopt either retrieval-oriented or reasoning-oriented paradigms to tackle this challenge, but both are constrained by two fundamental limitations:(1) They apply the same inference pipeline to all samples without accounting for uneven recognition difficulty, thereby leading to suboptimal accuracy and efficiency; (2) The lack of mechanisms to consolidate and reuse error-specific experience causes repeated failures on similar challenging cases. To address these limitations, we propose SARE, a Sample-wise Adaptive textbfREasoning framework for training-free FGVR. Specifically, SARE adopts a cascaded design that combines fast candidate retrieval with fine-grained reasoning, invoking the latter only when necessary. In the reasoning process, SARE incorporates a self-reflective experience mechanism that leverages past failures to provide transferable discriminative guidance during inference, without any parameter updates. Extensive experiments across 14 datasets substantiate that SARE achieves state-of-the-art performance while substantially reducing computational overhead.