GalaxAlign: Mimicking Citizen Scientists' Multimodal Guidance for Galaxy Morphology Analysis
Ruoqi Wang, Haitao Wang, Qiong Luo

TL;DR
GalaxAlign is a multimodal framework inspired by citizen scientists that aligns schematic symbols, textual labels, and galaxy images to improve galaxy classification and similarity search without extensive pretraining.
Contribution
It introduces a tri-modal alignment framework that leverages domain-specific multimodal knowledge, reducing reliance on costly pretraining for galaxy morphology analysis.
Findings
Effective fine-tuning of pre-trained models for astronomical tasks
Improved accuracy in galaxy classification and similarity search
Eliminates need for large annotated datasets
Abstract
Galaxy morphology analysis involves studying galaxies based on their shapes and structures. For such studies, fundamental tasks include identifying and classifying galaxies in astronomical images, as well as retrieving visually or structurally similar galaxies through similarity search. Existing methods either directly train domain-specific foundation models on large, annotated datasets or fine-tune vision foundation models on a smaller set of images. The former is effective but costly, while the latter is more resource-efficient but often yields lower accuracy. To address these challenges, we introduce GalaxAlign, a multimodal approach inspired by how citizen scientists identify galaxies in astronomical images by following textual descriptions and matching schematic symbols. Specifically, GalaxAlign employs a tri-modal alignment framework to align three types of data during…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMedical Image Segmentation Techniques · 3D Surveying and Cultural Heritage · Image Retrieval and Classification Techniques
MethodsSparse Evolutionary Training · Contrastive Learning · ALIGN
