Towards Object Segmentation Mask Selection Using Specular Reflections

TL;DR

This paper introduces a novel method for object segmentation that leverages specular reflections to improve mask accuracy without requiring additional training data, demonstrating significant performance gains over existing techniques.

Contribution

The proposed approach exploits the surface-located nature of specular reflections to enhance segmentation masks, outperforming state-of-the-art methods like SAM2 in synthetic and real images.

Findings

Up to 26.7% improvement in IoU over SAM2

22.3% increase in DSC

9.7% higher pixel accuracy

Abstract

Specular reflections pose a significant challenge for object segmentation, as their sharp intensity transitions often mislead both conventional algorithms and deep learning based methods. However, as the specular reflection must lie on the surface of the object, this fact can be exploited to improve the segmentation masks. By identifying the largest region containing the reflection as the object, we derive a more accurate object mask without requiring specialized training data or model adaption. We evaluate our method on both synthetic and real world images and compare it against established and state-of-the-art techniques including Otsu thresholding, YOLO, and SAM2. Compared to the best performing baseline SAM2, our approach achieves up to 26.7% improvement in IoU, 22.3% in DSC, and 9.7% in pixel accuracy. Qualitative evaluations on real world images further confirm the robustness and generalizability of the proposed approach.