Navigating Ideation Space: Decomposed Conceptual Representations for Positioning Scientific Ideas

TL;DR

This paper introduces the Ideation Space, a structured, multi-dimensional representation of scientific ideas that improves literature retrieval and novelty assessment, thereby accelerating scientific discovery.

Contribution

It proposes a novel decomposed, contrastively trained representation of scientific knowledge and a hierarchical retrieval and novelty assessment framework, addressing limitations of existing embedding and LLM approaches.

Findings

Achieves 16.7% improvement in Recall@30 over baselines

Attains 0.643 Hit Rate@30 in ideation transition retrieval

Correlates at 0.37 with expert judgments in novelty assessment

Abstract

Scientific discovery is a cumulative process and requires new ideas to be situated within an ever-expanding landscape of existing knowledge. An emerging and critical challenge is how to identify conceptually relevant prior work from rapidly growing literature, and assess how a new idea differentiates from existing research. Current embedding approaches typically conflate distinct conceptual aspects into single representations and cannot support fine-grained literature retrieval; meanwhile, LLM-based evaluators are subject to sycophancy biases, failing to provide discriminative novelty assessment. To tackle these challenges, we introduce the Ideation Space, a structured representation that decomposes scientific knowledge into three distinct dimensions, i.e., research problem, methodology, and core findings, each learned through contrastive training. This framework enables principled measurement of conceptual distance between ideas, and modeling of ideation transitions that capture the logical connections within a proposed idea. Building upon this representation, we propose a Hierarchical Sub-Space Retrieval framework for efficient, targeted literature retrieval, and a Decomposed Novelty Assessment algorithm that identifies which aspects of an idea are novel. Extensive experiments demonstrate substantial improvements, where our approach achieves Recall@30 of 0.329 (16.7% over baselines), our ideation transition retrieval reaches Hit Rate@30 of 0.643, and novelty assessment attains 0.37 correlation with expert judgments. In summary, our work provides a promising paradigm for future research on accelerating and evaluating scientific discovery.