From Boundaries to Semantics: Prompt-Guided Multi-Task Learning for Petrographic Thin-section Segmentation

TL;DR

This paper introduces Petro-SAM, a two-stage multi-task framework based on SAM, that effectively performs joint grain-edge and lithology segmentation on petrographic images by addressing domain gaps and multi-view integration.

Contribution

It proposes Petro-SAM, incorporating a Merge Block and multi-scale feature fusion to enable high-quality joint segmentation of petrographic images.

Findings

Achieves high-quality joint GES and LSS on petrographic images.

Effectively integrates multi-view polarized images.

Refines segmentation using multi-scale feature fusion and color-entropy priors.

Abstract

Grain-edge segmentation (GES) and lithology semantic segmentation (LSS) are two pivotal tasks for quantifying rock fabric and composition. However, these two tasks are often treated separately, and the segmentation quality is implausible albeit expensive, time-consuming, and expert-annotated datasets have been used. Recently, foundation models, especially the Segment Anything Model (SAM), have demonstrated impressive robustness for boundary alignment. However, directly adapting SAM to joint GES and LSS is nontrivial due to 1) severe domain gap induced by extinction-dependent color variations and ultra-fine grain boundaries, and 2) lacking novel modules for joint learning on multi-angle petrographic image stacks. In this paper, we propose Petro-SAM, a novel two-stage, multi-task framework that can achieve high-quality joint GES and LSS on petrographic images. Specifically, based on SAM, we introduce a Merge Block to integrate seven polarized views, effectively solving the extinction issue. Moreover, we introduce multi-scale feature fusion and color-entropy priors to refine the detection.