ModTrans: Translating Real-world Models for Distributed Training Simulator

TL;DR

ModTrans is a translator tool that converts real-world machine learning models into a format compatible with distributed training simulators like ASTRA-sim, enabling easier experimentation without physical resources.

Contribution

It introduces a format translator that supports importing any real-world model into ASTRA-sim, bridging a gap in existing distributed training simulators.

Findings

ModTrans's cost is negligible.

Supports importing any real-world model into ASTRA-sim.

Facilitates experiments without physical machines.

Abstract

Large-scale distributed training has been a research hot spot in machine learning systems for industry and academia in recent years. However, conducting experiments without physical machines and corresponding resources is difficult. One solution is to leverage distributed training simulators, but current ones like ASTRA-sim do not support importing real-world developed models, which poses challenges for ML researchers seeking to use them. Based on this challenge, we developed ModTrans, a translator supporting format translation from any real-world model to the ASTRA-sim simulator's input, removing the barrier between machine learning experts and machine learning system researchers. The experiment results show that ModTrans's cost is negligible.