Anomaly Object Segmentation with Vision-Language Models for Steel Scrap Recycling

TL;DR

This paper introduces a vision-language model fine-tuned for fine-grained anomaly detection in steel scrap recycling, aiming to improve impurity identification and reduce CO2 emissions.

Contribution

It presents a novel supervised fine-tuning approach of a vision-language model with multi-scale and text prompts for anomaly detection in steel scrap.

Findings

Effective anomaly detection at a fine-grained level

Improved impurity identification accuracy

Potential reduction in CO2 emissions

Abstract

Recycling steel scrap can reduce carbon dioxide (CO2) emissions from the steel industry. However, a significant challenge in steel scrap recycling is the inclusion of impurities other than steel. To address this issue, we propose vision-language-model-based anomaly detection where a model is finetuned in a supervised manner, enabling it to handle niche objects effectively. This model enables automated detection of anomalies at a fine-grained level within steel scrap. Specifically, we finetune the image encoder, equipped with multi-scale mechanism and text prompts aligned with both normal and anomaly images. The finetuning process trains these modules using a multiclass classification as the supervision.