SIFOTL: A Principled, Statistically-Informed Fidelity-Optimization Method for Tabular Learning

TL;DR

SIFOTL is a novel, privacy-preserving method for identifying data shift factors in tabular healthcare datasets, using summary statistics and twin models to outperform existing approaches in noisy, real-world scenarios.

Contribution

The paper introduces SIFOTL, a new approach that combines privacy-safe summaries, twin XGBoost models, and Pareto decision trees for robust, interpretable shift analysis in tabular data.

Findings

Achieves high F1 scores (0.85-0.96) in real-world healthcare datasets.

Outperforms existing methods like BigQuery Contribution Analysis and statistical tests.

Remains effective with observational noise, maintaining F1 >= 0.75.

Abstract

Identifying the factors driving data shifts in tabular datasets is a significant challenge for analysis and decision support systems, especially those focusing on healthcare. Privacy rules restrict data access, and noise from complex processes hinders analysis. To address this challenge, we propose SIFOTL (Statistically-Informed Fidelity-Optimization Method for Tabular Learning) that (i) extracts privacy-compliant data summary statistics, (ii) employs twin XGBoost models to disentangle intervention signals from noise with assistance from LLMs, and (iii) merges XGBoost outputs via a Pareto-weighted decision tree to identify interpretable segments responsible for the shift. Unlike existing analyses which may ignore noise or require full data access for LLM-based analysis, SIFOTL addresses both challenges using only privacy-safe summary statistics. Demonstrating its real-world efficacy, for a MEPS panel dataset mimicking a new Medicare drug subsidy, SIFOTL achieves an F1 score of 0.85, substantially outperforming BigQuery Contribution Analysis (F1=0.46) and statistical tests (F1=0.20) in identifying the segment receiving the subsidy. Furthermore, across 18 diverse EHR datasets generated based on Synthea ABM, SIFOTL sustains F1 scores of 0.86-0.96 without noise and >= 0.75 even with injected observational noise, whereas baseline average F1 scores range from 0.19-0.67 under the same tests. SIFOTL, therefore, provides an interpretable, privacy-conscious workflow that is empirically robust to observational noise.