TIP-Search: Time-Predictable Inference Scheduling for Market Prediction under Uncertain Load

TL;DR

TIP-Search is a scheduling framework that dynamically selects deep learning models for real-time financial market prediction, ensuring low latency and high accuracy under uncertain workloads.

Contribution

It introduces a novel online model selection method that profiles latency and accuracy offline, then applies it in real-time without needing input domain labels.

Findings

Outperforms static baselines with up to 8.5% accuracy improvement.

Achieves 100% deadline satisfaction in real-world datasets.

Effective in low-latency, uncertain financial inference environments.

Abstract

This paper proposes TIP-Search, a time-predictable inference scheduling framework for real-time market prediction under uncertain workloads. Motivated by the strict latency demands in high-frequency financial systems, TIP-Search dynamically selects a deep learning model from a heterogeneous pool, aiming to maximize predictive accuracy while satisfying per-task deadline constraints. Our approach profiles latency and generalization performance offline, then performs online task-aware selection without relying on explicit input domain labels. We evaluate TIP-Search on three real-world limit order book datasets (FI-2010, Binance BTC/USDT, LOBSTER AAPL) and demonstrate that it outperforms static baselines with up to 8.5% improvement in accuracy and 100% deadline satisfaction. Our results highlight the effectiveness of TIP-Search in robust low-latency financial inference under uncertainty.