DeepInnovator: Triggering the Innovative Capabilities of LLMs

TL;DR

DeepInnovator introduces a systematic training framework for LLMs to autonomously generate innovative research ideas by leveraging structured scientific knowledge and iterative idea prediction, significantly outperforming untrained models.

Contribution

This work presents a novel training paradigm and data pipeline that systematically enhances LLMs' ability to produce innovative research ideas, moving beyond prompt engineering.

Findings

DeepInnovator-14B outperforms untrained baselines with 80.53%-93.81% win rates.

The framework achieves performance comparable to leading LLMs.

Open-source dataset and code to foster community development.

Abstract

The application of Large Language Models (LLMs) in accelerating scientific discovery has garnered increasing attention, with a key focus on constructing research agents endowed with innovative capability, i.e., the ability to autonomously generate novel and significant research ideas. Existing approaches predominantly rely on sophisticated prompt engineering and lack a systematic training paradigm. To address this, we propose DeepInnovator, a training framework designed to trigger the innovative capability of LLMs. Our approach comprises two core components. (1) ``Standing on the shoulders of giants''. We construct an automated data extraction pipeline to extract and organize structured research knowledge from a vast corpus of unlabeled scientific literature. (2) ``Conjectures and refutations''. We introduce a ``Next Idea Prediction'' training paradigm, which models the generation of research ideas as an iterative process of continuously predicting, evaluating, and refining plausible and novel next idea. Both automatic and expert evaluations demonstrate that our DeepInnovator-14B significantly outperforms untrained baselines, achieving win rates of 80.53\%-93.81\%, and attains performance comparable to that of current leading LLMs. This work provides a scalable training pathway toward building research agents with genuine, originative innovative capability, and will open-source the dataset to foster community advancement. Source code and data are available at: https://github.com/HKUDS/DeepInnovator.