AI Technical Considerations: Data Storage, Cloud usage and AI Pipeline

TL;DR

This paper discusses the technical aspects of data storage, cloud computing, and AI pipelines essential for developing and deploying AI systems, emphasizing standards, legal considerations, and hybrid infrastructure approaches.

Contribution

It provides a comprehensive overview of technical considerations for AI environments, focusing on data management, cloud integration, and pipeline implementation.

Findings

Highlights importance of standardized data collection and legal compliance.

Explains hybrid AI pipeline architectures combining on-premise and cloud.

Offers technical guidance for designing AI infrastructure.

Abstract

Artificial intelligence (AI), especially deep learning, requires vast amounts of data for training, testing, and validation. Collecting these data and the corresponding annotations requires the implementation of imaging biobanks that provide access to these data in a standardized way. This requires careful design and implementation based on the current standards and guidelines and complying with the current legal restrictions. However, the realization of proper imaging data collections is not sufficient to train, validate and deploy AI as resource demands are high and require a careful hybrid implementation of AI pipelines both on-premise and in the cloud. This chapter aims to help the reader when technical considerations have to be made about the AI environment by providing a technical background of different concepts and implementation aspects involved in data storage, cloud usage, and AI pipelines.