The LSST Data Management System

TL;DR

The LSST Data Management System is a comprehensive, scalable software stack designed to handle massive data volumes from the LSST survey, enabling real-time analysis and multiple data releases over its operational lifetime.

Contribution

This paper introduces a new open-source, high-performance data processing software stack tailored for the LSST, capable of processing data from various existing surveys.

Findings

Prototype processing of SDSS, DECam, MegaCam data

Supports real-time alert stream generation

Enables multiple data releases over 10 years

Abstract

The Large Synoptic Survey Telescope (LSST) is a large-aperture, wide-field, ground-based survey system that will image the sky in six optical bands from 320 to 1050 nm, uniformly covering approximately $18,000$deg$^2$ of the sky over 800 times. The LSST is currently under construction on Cerro Pach\'on in Chile, and expected to enter operations in 2022. Once operational, the LSST will explore a wide range of astrophysical questions, from discovering "killer" asteroids to examining the nature of Dark Energy. The LSST will generate on average 15 TB of data per night, and will require a comprehensive Data Management system to reduce the raw data to scientifically useful catalogs and images with minimum human intervention. These reductions will result in a real-time alert stream, and eleven data releases over the 10-year duration of LSST operations. To enable this processing, the LSST project is developing a new, general-purpose, high-performance, scalable, well documented, open source data processing software stack for O/IR surveys. Prototypes of this stack are already capable of processing data from existing cameras (e.g., SDSS, DECam, MegaCam), and form the basis of the Hyper-Suprime Cam (HSC) Survey data reduction pipeline.