Leveraging XP and CRISP-DM for Agile Data Science Projects

TL;DR

This paper investigates how integrating XP and CRISP-DM methodologies can enhance agility and collaboration in Data Science projects, demonstrated through a case study at Elo7 with positive adoption results.

Contribution

It presents a practical approach to combining XP and CRISP-DM in Data Science, supported by empirical data from a real-world case study.

Findings

86% of team members frequently use CRISP-DM

71% adopt XP practices in projects

Successful integration of XP and CRISP-DM enhances project agility

Abstract

This study explores the integration of eXtreme Programming (XP) and the Cross-Industry Standard Process for Data Mining (CRISP-DM) in agile Data Science projects. We conducted a case study at the e-commerce company Elo7 to answer the research question: How can the agility of the XP method be integrated with CRISP-DM in Data Science projects? Data was collected through interviews and questionnaires with a Data Science team consisting of data scientists, ML engineers, and data product managers. The results show that 86% of the team frequently or always applies CRISP-DM, while 71% adopt XP practices in their projects. Furthermore, the study demonstrates that it is possible to combine CRISP-DM with XP in Data Science projects, providing a structured and collaborative approach. Finally, the study generated improvement recommendations for the company.