catsHTM - A tool for fast accessing and cross-matching large astronomical catalogs

TL;DR

catsHTM is a software tool that enables rapid local access and cross-matching of large astronomical catalogs by using hierarchical data structures and efficient search algorithms, facilitating faster data analysis in astronomy.

Contribution

The paper introduces catsHTM, a novel tool that reformats large astronomical catalogs into HDF5 files with hierarchical indexing for ultra-fast local cone searches and cross-matching.

Findings

Supports numerous major astronomical catalogs.

Performs cone searches within milliseconds.

Provides multi-language code for data access and analysis.

Abstract

Fast access to large catalogs is required for some astronomical applications. Here we introduce the catsHTM tool, consisting of several large catalogs reformatted into HDF5-based file format, which can be downloaded and used locally. To allow fast access, the catalogs are partitioned into hierarchical triangular meshes and stored in HDF5 files. Several tools are provided to perform efficient cone searches at resolutions spanning from a few arc seconds to degrees, within a few milliseconds time. The first released version includes the following catalogs (by alphabetical order): 2MASS, 2MASS extended sources, AKARI, APASS, Cosmos, DECaLS/DR5, FIRST, GAIA/DR1, GAIA/DR2, GALEX/DR6Plus7, HSC/v2, IPHAS/DR2, NED redshifts, NVSS, Pan-STARRS1/DR1, PTF photometric catalog, ROSAT faint source, SDSS sources, SDSS/DR14 spectroscopy, Spitzer/SAGE, Spitzer/IRAC galactic center, UCAC4, UKIDSS/DR10, VST/ATLAS/DR3, VST/KiDS/DR3, WISE and XMM. We provide Python code that allows to perform cone searches, as well as MATLAB code for performing cone searches, catalog cross-matching, general searches, as well as load and create these catalogs.