SHREC 2021: Classification in cryo-electron tomograms

TL;DR

This paper introduces a new simulated dataset for benchmarking protein localization and classification in cryo-electron tomograms, demonstrating that learning-based methods outperform traditional template matching.

Contribution

The paper provides a novel simulated dataset and a comprehensive evaluation of multiple computational methods for cryo-ET analysis, highlighting the advantages of learning-based approaches.

Findings

Learning-based methods outperform template matching in localization and classification.

Smaller particles are more challenging for all methods.

The dataset enables standardized benchmarking of cryo-ET computational techniques.

Abstract

Cryo-electron tomography (cryo-ET) is an imaging technique that allows three-dimensional visualization of macro-molecular assemblies under near-native conditions. Cryo-ET comes with a number of challenges, mainly low signal-to-noise and inability to obtain images from all angles. Computational methods are key to analyze cryo-electron tomograms. To promote innovation in computational methods, we generate a novel simulated dataset to benchmark different methods of localization and classification of biological macromolecules in tomograms. Our publicly available dataset contains ten tomographic reconstructions of simulated cell-like volumes. Each volume contains twelve different types of complexes, varying in size, function and structure. In this paper, we have evaluated seven different methods of finding and classifying proteins. Seven research groups present results obtained with learning-based methods and trained on the simulated dataset, as well as a baseline template matching (TM), a traditional method widely used in cryo-ET research. We show that learning-based approaches can achieve notably better localization and classification performance than TM. We also experimentally confirm that there is a negative relationship between particle size and performance for all methods.