Optimizing the AI Development Process by Providing the Best Support Environment

TL;DR

This paper investigates data management challenges in AI/ML development, especially data scarcity, and proposes a Python-based framework utilizing deep learning data augmentation techniques to enhance data quantity and quality.

Contribution

It introduces a novel framework that applies deep learning data augmentation to address data scarcity in ML development, improving model performance.

Findings

The framework effectively increases data volume for ML models.

Data augmentation improves model accuracy in data-scarce environments.

The approach is applicable to confidential data scenarios.

Abstract

The purpose of this study is to investigate the development process for Artificial inelegance (AI) and machine learning (ML) applications in order to provide the best support environment. The main stages of ML are problem understanding, data management, model building, model deployment and maintenance. This project focuses on investigating the data management stage of ML development and its obstacles as it is the most important stage of machine learning development because the accuracy of the end model is relying on the kind of data fed into the model. The biggest obstacle found on this stage was the lack of sufficient data for model learning, especially in the fields where data is confidential. This project aimed to build and develop a framework for researchers and developers that can help solve the lack of sufficient data during data management stage. The framework utilizes several data augmentation techniques that can be used to generate new data from the original dataset which can improve the overall performance of the ML applications by increasing the quantity and quality of available data to feed the model with the best possible data. The framework was built using python language to perform data augmentation using deep learning advancements.