Lighting-aware Unified Model for Instance Segmentation

TL;DR

This paper introduces Lighting Convolutional-Attention (LCA), a novel adapter module that improves the robustness of instance segmentation models under diverse lighting conditions without retraining the entire backbone.

Contribution

The work proposes LCA, a dual-branch architecture with a pairwise training strategy, to enhance lighting robustness in instance segmentation models without fine-tuning heavy backbones.

Findings

LCA significantly improves segmentation performance under varied illumination.

The synthetic Unity-based dataset accurately simulates complex lighting for benchmarking.

Experimental results show superior lighting-robust segmentation compared to existing methods.

Abstract

Foundation models like the Segment Anything Model (SAM) demonstrate impressive zero-shot generalization but frequently degrade under diverse real-world illumination, particularly for instance segmentation. In this work, we address this limitation by developing \textit{Lighting Convolutional-Attention (\lca{})}, an adapter module that enhances segmentation robustness without fine-tuning the heavy backbone. \lca{} employs a dual-branch architecture to process RGB features alongside contrast maps, enabling physically motivated sensitivity to structural changes rather than illumination artifacts. We optimize \lca{} through a pairwise training strategy, introducing a targeted loss term that explicitly penalizes discrepancies between clean images and their corresponding illumination variants. To evaluate and support this architecture, we conduct a comprehensive empirical study across multiple existing benchmarks and present a novel Unity-based synthetic dataset specifically designed to accurately replicate complex real-world lighting conditions. Extensive experimental results demonstrate that our approach successfully bridges the domain gap, delivering superior lighting-robust segmentation.