TDLight: A Framework for Incremental Light Curve Management and Smart Classification
Xinghang Yu, Ce Yu, Zeguang Shao, Bin Yang

TL;DR
TDLight is a scalable, real-time system that manages and classifies astronomical light curves incrementally, enabling timely analysis and discovery in large-scale time-domain surveys.
Contribution
It introduces a unified framework combining high-performance incremental storage with real-time classification tailored for astronomical light curves.
Findings
Achieves high-throughput data ingestion up to 954,000 rows/sec
Supports efficient cone-search queries with HEALPix indexing
Enables early classification and identification of high-value transient candidates
Abstract
With the exponential growth of time-domain surveys, the volume of light curves has increased rapidly. However, many survey projects, such as Gaia, still rely on offline batch-processing workflows in which data are calibrated, merged, and released only after an observing phase is completed. This latency delays scientific analysis and causes many high-value transient events to be buried in archival data, missing the window for timely follow-up. While existing alert brokers handle heterogeneous data streams, it remains difficult to deploy a unified framework that combines high-performance incremental storage with real-time classification on local infrastructure. To address this challenge, we propose TDLight, a scalable system that adapts the time-series database TDengine (a high-performance IoT database) for astronomical data using a one-table-per-source schema. This architecture supports…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTime Series Analysis and Forecasting · Data Stream Mining Techniques · Data Management and Algorithms
