Evaluating Generic Auto-ML Tools for Computational Pathology

TL;DR

This study evaluates the effectiveness of generic AutoML tools for neural network architecture search and hyperparameter optimization in computational pathology, finding they perform comparably to manual methods with less effort.

Contribution

It provides an empirical assessment of AutoML tools' performance in histological image classification tasks, highlighting their viability as alternatives to manual optimization.

Findings

Default AutoML CNN architectures match published results.

Hyperparameter tuning offers limited performance gains.

Classifier performance varies due to non-deterministic effects.

Abstract

Image analysis tasks in computational pathology are commonly solved using convolutional neural networks (CNNs). The selection of a suitable CNN architecture and hyperparameters is usually done through exploratory iterative optimization, which is computationally expensive and requires substantial manual work. The goal of this article is to evaluate how generic tools for neural network architecture search and hyperparameter optimization perform for common use cases in computational pathology. For this purpose, we evaluated one on-premises and one cloud-based tool for three different classification tasks for histological images: tissue classification, mutation prediction, and grading. We found that the default CNN architectures and parameterizations of the evaluated AutoML tools already yielded classification performance on par with the original publications. Hyperparameter optimization for these tasks did not substantially improve performance, despite the additional computational effort. However, performance varied substantially between classifiers obtained from individual AutoML runs due to non-deterministic effects. Generic CNN architectures and AutoML tools could thus be a viable alternative to manually optimizing CNN architectures and parametrizations. This would allow developers of software solutions for computational pathology to focus efforts on harder-to-automate tasks such as data curation.