Let the Abyss Stare Back Adaptive Falsification for Autonomous Scientific Discovery

TL;DR

This paper introduces DASES, a framework for adaptive falsification in autonomous scientific discovery, actively challenging candidate models to improve validation and discovery robustness.

Contribution

It presents a novel co-evolutionary framework involving an Innovator, Abyss Falsifier, and Causal Extractor for more rigorous scientific artifact validation.

Findings

DASES rejects artifacts that static validation would accept.

It identifies candidates surviving the falsification frontier.

FNG-CE loss outperforms CE and CE+L2 in benchmarks including ImageNet.

Abstract

Autonomous scientific discovery is entering a more dangerous regime: once the evaluator is frozen, a sufficiently strong search process can learn to win the exam without learning the mechanism the task was meant to reveal. This is the idea behind our title. To let the abyss stare back is to make evaluation actively push against the candidate through adaptive falsification, rather than passively certify it through static validation. We introduce DASES, a falsification-driven framework in which an Innovator, an Abyss Falsifier, and a Mechanistic Causal Extractor co-evolve executable scientific artifacts and scientifically admissible counterexample environments under a fixed scientific contract. In a controlled loss-discovery problem with a single editable locus, DASES rejects artifacts that static validation would have accepted, identifies the first candidate that survives the admissible falsification frontier, and discovers FNG-CE, a loss that transfers beyond the synthetic discovery environment and consistently outperforms CE and CE+L2 under controlled comparisons across standard benchmarks, including ImageNet.