Faster or Slower? Performance Mystery of Python Idioms Unveiled with Empirical Evidence

TL;DR

This paper systematically investigates the performance impacts of Python idioms through large-scale empirical experiments, providing evidence-based insights and practical guidelines for developers.

Contribution

It presents a large-scale empirical study comparing idiomatic and non-idiomatic Python code, revealing performance effects and root causes at the bytecode level.

Findings

Identifies performance discrepancies between synthetic and real-world code.

Analyzes code features influencing performance changes.

Provides actionable suggestions for idiom usage based on empirical evidence.

Abstract

The usage of Python idioms is popular among Python developers in a formative study of 101 performance-related questions of Python idioms on Stack Overflow, we find that developers often get confused about the performance impact of Python idioms and use anecdotal toy code or rely on personal project experience which is often contradictory in performance outcomes. There has been no large-scale, systematic empirical evidence to reconcile these performance debates. In the paper, we create a large synthetic dataset with 24,126 pairs of non-idiomatic and functionally-equivalent idiomatic code for the nine unique Python idioms identified in Zhang et al., and reuse a large real-project dataset of 54,879 such code pairs provided by Zhang et al. We develop a reliable performance measurement method to compare the speedup or slowdown by idiomatic code against non-idiomatic counterpart, and analyze the performance discrepancies between the synthetic and real-project code, the relationships between code features and performance changes, and the root causes of performance changes at the bytecode level. We summarize our findings as some actionable suggestions for using Python idioms.