JVLGS: Joint Vision-Language Gas Leak Segmentation

TL;DR

JVLGS is a novel framework that combines visual and textual data to improve gas leak segmentation accuracy, effectively reducing false positives and outperforming existing methods across various scenarios and learning settings.

Contribution

The paper introduces JVLGS, a joint vision-language model that enhances gas leak segmentation by integrating multimodal data and incorporating a post-processing step to minimize false positives.

Findings

JVLGS significantly outperforms state-of-the-art methods.

The model performs well in both supervised and few-shot settings.

It effectively reduces false positives caused by noise and non-target objects.

Abstract

Gas leaks pose serious threats to human health and contribute significantly to atmospheric pollution, drawing increasing public concern. However, the lack of effective detection methods hampers timely and accurate identification of gas leaks. While some vision-based techniques leverage infrared videos for leak detection, the blurry and non-rigid nature of gas clouds often limits their effectiveness. To address these challenges, we propose a novel framework called Joint Vision-Language Gas leak Segmentation (JVLGS), which integrates the complementary strengths of visual and textual modalities to enhance gas leak representation and segmentation. Recognizing that gas leaks are sporadic and many video frames may contain no leak at all, our method incorporates a post-processing step to reduce false positives caused by noise and non-target objects, an issue that affects many existing approaches. Extensive experiments conducted across diverse scenarios show that JVLGS significantly outperforms state-of-the-art gas leak segmentation methods. We evaluate our model under both supervised and few-shot learning settings, and it consistently achieves strong performance in both, whereas competing methods tend to perform well in only one setting or poorly in both. Code available at: https://github.com/GeekEagle/JVLGS