LMC: Large Model Collaboration with Cross-assessment for Training-Free Open-Set Object Recognition

TL;DR

The paper introduces LMC, a training-free framework that leverages collaboration among diverse large pre-trained models to improve open-set object recognition by extracting implicit knowledge without additional training.

Contribution

It proposes a novel training-free collaboration framework among large models, incorporating new designs to extract implicit knowledge for open-set recognition.

Findings

Effective in reducing reliance on spurious features

Improves open-set recognition accuracy

Works with off-the-shelf large models

Abstract

Open-set object recognition aims to identify if an object is from a class that has been encountered during training or not. To perform open-set object recognition accurately, a key challenge is how to reduce the reliance on spurious-discriminative features. In this paper, motivated by that different large models pre-trained through different paradigms can possess very rich while distinct implicit knowledge, we propose a novel framework named Large Model Collaboration (LMC) to tackle the above challenge via collaborating different off-the-shelf large models in a training-free manner. Moreover, we also incorporate the proposed framework with several novel designs to effectively extract implicit knowledge from large models. Extensive experiments demonstrate the efficacy of our proposed framework. Code is available https://github.com/Harryqu123/LMC