SPOCK 2.0: Update to the FeatureClassifier in the Stability of Planetary Orbital Configurations Klassifier

TL;DR

This paper improves the SPOCK 2.0 classifier for planetary system stability prediction by using system-specific timescales, correcting dataset errors, and releasing an updated model and dataset.

Contribution

The authors enhance SPOCK's stability classifier with system-specific timescales, fix dataset duplication issues, and provide a new trained model and dataset release.

Findings

Improved AUC from 0.943 to 0.950 with system-specific timescales.

Identified and corrected dataset duplication and misclassification issues.

Released a cleaned dataset of over 100,000 integrations and an updated classifier.

Abstract

The Stability of Planetary Orbital Configurations Klassifier (SPOCK) package collects machine learning models for predicting the stability and collisional evolution of compact planetary systems. In this paper we explore improvements to SPOCK's binary stability classifier (FeatureClassifier), which predicts orbital stability by collecting data over a short N-body integration of a system. We find that by using a system-specific timescale (rather than a fixed $10^4$ orbits) for the integration, and by using this timescale as an additional feature, we modestly improve the model's AUC metric from 0.943 to 0.950 (AUC=1 for a perfect model). We additionally discovered that $\approx 10\%$ of N-body integrations in SPOCK's original training dataset were duplicated by accident, and that $<1\%$ were misclassified as stable when they in fact led to ejections. We provide a cleaned dataset of 100,000+ unique integrations, release a newly trained stability classification model, and make minor updates to the API.