Deep Research of Deep Research: From Transformer to Agent, From AI to AI for Science

TL;DR

This paper explores the evolution of large language models and generative AI from simple models to complex agents, emphasizing their role in scientific discovery and the integration of AI and science.

Contribution

It provides a comprehensive framework unifying industry and academia perspectives on deep research and AI for Science, outlining a developmental roadmap from Transformers to agents.

Findings

LLMs and Stable Diffusion are key pillars of generative AI.

Deep research aims to surpass top human scientists in scientific discovery.

The paper identifies major challenges and future research directions in AI for Science.

Abstract

With the advancement of large language models (LLMs) in their knowledge base and reasoning capabilities, their interactive modalities have evolved from pure text to multimodality and further to agentic tool use. Consequently, their applications have broadened from question answering to AI assistants and now to general-purpose agents. Deep research (DR) represents a prototypical vertical application for general-purpose agents, which represents an ideal approach for intelligent information processing and assisting humans in discovering and solving problems, with the goal of reaching or even surpassing the level of top human scientists. This paper provides a deep research of deep research. We articulate a clear and precise definition of deep research and unify perspectives from industry's deep research and academia's AI for Science (AI4S) within a developmental framework. We position LLMs and Stable Diffusion as the twin pillars of generative AI, and lay out a roadmap evolving from the Transformer to agents. We examine the progress of AI4S across various disciplines. We identify the predominant paradigms of human-AI interaction and prevailing system architectures, and discuss the major challenges and fundamental research issues that remain. AI supports scientific innovation, and science also can contribute to AI growth (Science for AI, S4AI). We hope this paper can help bridge the gap between the AI and AI4S communities.