Harnessing Pre-Resolution Signals for Future Prediction Agents

TL;DR

This paper introduces Milkyway, a future prediction agent that leverages evolving evidence and repeated forecasts to extract pre-resolution signals, improving predictions before outcomes are known.

Contribution

The paper presents a novel approach using pre-resolution signals and a persistent harness to enhance future prediction accuracy through iterative evidence revisits.

Findings

Milkyway outperforms baseline models on FutureX and FutureWorld benchmarks.

Pre-resolution signals significantly improve forecast updates before outcome resolution.

Harness evolution driven by pre-resolution signals is key to performance gains.

Abstract

Many high-stakes decisions depend on forecasts made before outcomes are known. In this future prediction setting, the central challenge is that public evidence evolves over time, while the main supervision signal arrives only after resolution: the realized outcome mainly assesses final correctness, offering only coarse guidance on what to track, what to verify, and which judgments to leave uncertain along the way. Our key observation is that revisiting the same unresolved question over time creates informative temporal contrasts across evolving evidence and repeated forecasts, exposing what earlier attempts missed before resolution and yielding a diagnostic signal we call the pre-resolution signal. We instantiate this idea in Milkyway, a future prediction agent with a persistent future prediction harness, an editable external state that stores reusable procedural guidance across revisits to the same unresolved question. As the same unresolved question is revisited, Milkyway extracts pre-resolution signals from evolving evidence and repeated forecasts, uses them to update the harness, and improves later forecasts on that question before resolution. After resolution, the realized outcome serves as a post-resolution check of provisional updates. On the FutureX and FutureWorld benchmarks, Milkyway achieves strong performance against competitive baselines, and a mechanism study suggests that the gains stem from harness evolution driven by pre-resolution signals rather than repeated prediction alone.