At FullTilt: Real-Time Open-Set 3D Macromolecule Detection Directly from Tilted 2D Projections

TL;DR

FullTilt is an innovative framework for real-time open-set 3D macromolecule detection from tilt-series in cryo-electron tomography, significantly reducing computational costs and enabling large-scale analysis.

Contribution

The paper introduces FullTilt, a novel end-to-end method that operates directly on 2D tilt-series, improving speed and efficiency over traditional volumetric approaches.

Findings

Achieves state-of-the-art zero-shot detection performance.

Reduces runtime and VRAM usage by orders of magnitude.

Enables large-scale, rapid proteomics analysis.

Abstract

Open-set 3D macromolecule detection in cryogenic electron tomography eliminates the need for target-specific model retraining. However, strict VRAM constraints prohibit processing an entire 3D tomogram, forcing current methods to rely on slow sliding-window inference over extracted subvolumes. To overcome this, we propose FullTilt, an end-to-end framework that redefines 3D detection by operating directly on aligned 2D tilt-series. Because a tilt-series contains significantly fewer images than slices in a reconstructed tomogram, FullTilt eliminates redundant volumetric computation, accelerating inference by orders of magnitude. To process the entire tilt-series simultaneously, we introduce a tilt-series encoder to efficiently fuse cross-view information. We further propose a multiclass visual prompt encoder for flexible prompting, a tilt-aware query initializer to effectively anchor 3D queries, and an auxiliary geometric primitives module to enhance the model's understanding of multi-view geometry while improving robustness to adverse imaging artifacts. Extensive evaluations on three real-world datasets demonstrate that FullTilt achieves state-of-the-art zero-shot performance while drastically reducing runtime and VRAM requirements, paving the way for rapid, large-scale visual proteomics analysis. All code and data will be publicly available upon publication.