qlty: handling large tensors in scientific imaging

TL;DR

qlty is a toolkit that enables efficient handling of large volumetric datasets in scientific imaging by providing tensor management techniques for subsampling, cleaning, and stitching, thus facilitating deep learning in resource-constrained environments.

Contribution

The paper introduces qlty, a novel toolkit that addresses the challenge of processing large-scale scientific imaging data with limited computational resources.

Findings

Enables effective training on large datasets

Supports subsampling, cleaning, and stitching of tensors

Facilitates deep learning in resource-limited environments

Abstract

In scientific imaging, deep learning has become a pivotal tool for image analytics. However, handling large volumetric datasets, which often exceed the memory capacity of standard GPUs, require special attention when subjected to deep learning efforts. This paper introduces qlty, a toolkit designed to address these challenges through tensor management techniques. qlty offers robust methods for subsampling, cleaning, and stitching of large-scale spatial data, enabling effective training and inference even in resource-limited environments.