A Critical Look at the Current Usage of Foundation Model for Dense Recognition Task

TL;DR

This paper critically examines the application of foundation models in dense recognition tasks, highlighting current limitations and providing experimental insights to guide future research in deploying these models effectively.

Contribution

It offers a survey of existing methods for dense recognition using foundation models and presents preliminary analysis of diffusion-based segmentation approaches.

Findings

Current diffusion-based segmentation methods are not optimal.

Foundation models show promise but face challenges in dense recognition tasks.

Insights provided aim to improve future deployment strategies.

Abstract

In recent years large model trained on huge amount of cross-modality data, which is usually be termed as foundation model, achieves conspicuous accomplishment in many fields, such as image recognition and generation. Though achieving great success in their original application case, it is still unclear whether those foundation models can be applied to other different downstream tasks. In this paper, we conduct a short survey on the current methods for discriminative dense recognition tasks, which are built on the pretrained foundation model. And we also provide some preliminary experimental analysis of an existing open-vocabulary segmentation method based on Stable Diffusion, which indicates the current way of deploying diffusion model for segmentation is not optimal. This aims to provide insights for future research on adopting foundation model for downstream task.