From Code Smells to Best Practices: Tackling Resource Leaks in PyTorch, TensorFlow, and Keras

TL;DR

This paper systematically identifies code smells causing resource leaks in popular ML frameworks and provides best practices to enhance resource efficiency and sustainability in ML applications.

Contribution

First comprehensive study analyzing resource leak code smells across PyTorch, TensorFlow, and Keras, with actionable best practices for mitigation.

Findings

Identified 30 PyTorch-related resource leak smells.

Identified 16 TensorFlow/Keras-related resource leak smells.

Developed 50 recommended coding patterns to reduce resource leaks.

Abstract

Much of the existing ML research focuses on model performance metrics, leaving limited attention to the long-term sustainability and resource efficiency of ML applications. While high performance is essential, ensuring efficient resource management is equally critical for robust deployment. This study addresses this gap by systematically identifying code smells that lead to resource leaks in ML applications. We conducted an empirical investigation of developer discussions and real-world code snippets from PyTorch, TensorFlow, and Keras. The analysis identified 30 PyTorch-related smells and 16 TensorFlow/Keras smells linked to resource leaks. These smells were categorized in two ways: (1) based on their root causes, and (2) as general ML smells with framework-specific characteristics. For each smell, we derived at least one best practice, resulting in 50 recommended coding patterns aimed at reducing resource leakage and improving efficiency. To ensure the validity of our findings, we employed a three-phase validation process involving independent analysis by three authors followed by consensus discussions. This is the first comprehensive study to examine resource-leak-inducing code smells across major ML frameworks and to present actionable best practices for mitigating them. The contributions support developers in building more efficient and sustainable ML applications and offer a structured view of the underlying causes of resource leaks.