Foundation Models for Zero-Shot Segmentation of Scientific Images without AI-Ready Data

TL;DR

Zenesis is a no-code platform that enables zero-shot segmentation of scientific images by integrating lightweight multimodal adaptation, human-in-the-loop refinement, and heuristic enhancement, significantly outperforming traditional methods.

Contribution

The paper introduces Zenesis, a novel no-code platform that improves zero-shot segmentation of scientific images without requiring AI-ready data, combining multimodal adaptation and user interaction.

Findings

Achieved high accuracy and IoU on FIB-SEM datasets.

Outperformed traditional methods like Otsu and SAM.

Demonstrated effectiveness in domains with limited annotated data.

Abstract

Zero-shot and prompt-based models have excelled at visual reasoning tasks by leveraging large-scale natural image corpora, but they often fail on sparse and domain-specific scientific image data. We introduce Zenesis, a no-code interactive computer vision platform designed to reduce data readiness bottlenecks in scientific imaging workflows. Zenesis integrates lightweight multimodal adaptation for zero-shot inference on raw scientific data, human-in-the-loop refinement, and heuristic-based temporal enhancement. We validate our approach on Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) datasets of catalyst-loaded membranes. Zenesis outperforms baselines, achieving an average accuracy of 0.947, Intersection over Union (IoU) of 0.858, and Dice score of 0.923 on amorphous catalyst samples; and 0.987 accuracy, 0.857 IoU, and 0.923 Dice on crystalline samples. These results represent a significant performance gain over conventional methods such as Otsu thresholding and standalone models like the Segment Anything Model (SAM). Zenesis enables effective image segmentation in domains where annotated datasets are limited, offering a scalable solution for scientific discovery.