A Study of Adaptive Modeling Towards Robust Generalization

TL;DR

This paper introduces an adaptive all-atom framework for large language models that enhances reasoning over biomolecular structures by explicitly incorporating geometric cues and dynamically allocating structural tokens, improving grounding and reducing hallucinations.

Contribution

The proposed method unifies geometric information with language reasoning and adaptively scales structural tokens, addressing limitations of fixed-length encodings and rigid fusion in previous models.

Findings

Consistent performance improvements across all-atom benchmarks.

Enhanced structure grounding and reduced hallucinations.

Effective complexity-aware token allocation.

Abstract

Large language models (LLMs) increasingly support reasoning over biomolecular structures, but most existing approaches remain modality-specific and rely on either sequence-style encodings or fixed-length connector tokens for structural inputs. These designs can under-expose explicit geometric cues and impose rigid fusion bottlenecks, leading to over-compression and poor token allocation as structural complexity grows. We present a unified all-atom framework that grounds language reasoning in geometric information while adaptively scaling structural tokens. The method first constructs variable-size structural patches on molecular graphs using an instruction-conditioned gating policy, enabling complexity-aware allocation of query tokens. It then refines the resulting patch tokens via cross-attention with modality embeddings and injects geometry-informed tokens into the language model to improve structure grounding and reduce structural hallucinations. Across diverse all-atom benchmarks, the proposed approach yields consistent gains in heterogeneous structure-grounded reasoning. An anonymized implementation is provided in the supplementary material.