A Command-line Cross-matching tool for modern astrophysical pipelines

TL;DR

This paper introduces C3, a high-performance, flexible command-line tool for efficiently cross-matching massive astronomical catalogs, suitable for integration into various data analysis pipelines in the era of petabyte-scale data.

Contribution

The paper presents the architecture and features of C3, a scalable, multi-platform command-line tool for catalog cross-matching, including a customizable version for specific astrophysical applications.

Findings

C3 achieves high performance with multi-core parallel processing.

The tool is flexible for standalone or pipeline integration.

A customized version was successfully used in the ViaLactea project.

Abstract

The emerging need for efficient, reliable and scalable astronomical catalog cross-matching is becoming more pressing in the current data-driven science era, where the size of data has rapidly increased up to the Petabyte scale. C3 (Command-line Catalogue Cross-matching) is a multi-platform tool designed to efficiently cross-match massive catalogues from modern astronomical surveys, ensuring high-performance capabilities through the use of a multi-core parallel processing paradigm. The tool has been conceived to be executed as a stand-alone command-line process or integrated within any generic data reduction/analysis pipeline, providing the maximum flexibility to the end user, in terms of parameter configuration, coordinates and cross-matching types. In this work we present the architecture and the features of the tool. Moreover, since the modular design of the tool enables an easy customization to specific use cases and requirements, we present also an example of a customized C3 version designed and used in the FP7 project ViaLactea, dedicated to cross-correlate Hi-GAL clumps with multi-band compact sources.