Porting the LSST Data Management Pipeline Software to Python 3

TL;DR

This paper discusses the process and challenges of porting the LSST data management pipelines from Python 2 to Python 3, ensuring compatibility during a transitional period.

Contribution

It presents a practical approach using the Python future package to support both Python 2 and 3 during migration of large scientific software.

Findings

Porting involved over 100,000 lines of code.

Using the future package facilitated dual support for Python 2 and 3.

Encountered significant compatibility and testing challenges.

Abstract

The LSST data management science pipelines software consists of more than 100,000 lines of Python 2 code. LSST operations will begin after support for Python 2 has been dropped by the Python community in 2020, and we must therefore plan to migrate the codebase to Python 3. During the transition period we must also support our community of active Python 2 users and this complicates the porting significantly. We have decided to use the Python future package as the basis for our port to enable support for Python 2 and Python 3 simultaneously, whilst developing with a mindset more suited to Python 3. In this paper we report on the current status of the port and the difficulties that have been encountered.