Vision-Language Models for Infrared Industrial Sensing in Additive Manufacturing Scene Description

TL;DR

This paper presents VLM-IRIS, a zero-shot framework that adapts vision-language models to infrared industrial sensing, enabling accurate thermal scene understanding without retraining, useful for low-light manufacturing environments.

Contribution

The work introduces a novel method to preprocess infrared images for compatibility with existing VLMs, extending their zero-shot capabilities to thermal imaging in industrial settings.

Findings

Achieved high accuracy in infrared workpiece detection without retraining.

Demonstrated effective zero-shot thermal scene understanding in manufacturing.

Extended VLM applicability to infrared data with preprocessing techniques.

Abstract

Many manufacturing environments operate in low-light conditions or within enclosed machines where conventional vision systems struggle. Infrared cameras provide complementary advantages in such environments. Simultaneously, supervised AI systems require large labeled datasets, which makes zero-shot learning frameworks more practical for applications including infrared cameras. Recent advances in vision-language foundation models (VLMs) offer a new path in zero-shot predictions from paired image-text representations. However, current VLMs cannot understand infrared camera data since they are trained on RGB data. This work introduces VLM-IRIS (Vision-Language Models for InfraRed Industrial Sensing), a zero-shot framework that adapts VLMs to infrared data by preprocessing infrared images captured by a FLIR Boson sensor into RGB-compatible inputs suitable for CLIP-based encoders. We demonstrate zero-shot workpiece presence detection on a 3D printer bed where temperature differences between the build plate and workpieces make the task well-suited for thermal imaging. VLM-IRIS converts the infrared images to magma representation and applies centroid prompt ensembling with a CLIP ViT-B/32 encoder to achieve high accuracy on infrared images without any model retraining. These findings demonstrate that the proposed improvements to VLMs can be effectively extended to thermal applications for label-free monitoring.