Celeritas: Fast Optimizer for Large Dataflow Graphs
Hengwei Xu, Yong Liao, Haiyong Xie, Pengyuan Zhou

TL;DR
Celeritas is a fast framework that optimizes device placement for large neural network models, significantly reducing policy generation time and improving model training efficiency.
Contribution
It introduces a novel, efficient device placement optimization framework that outperforms existing methods in speed and model performance.
Findings
Reduces placement policy generation time by 26.4%.
Improves model running time by 34.2%.
Effective for large models in distributed training.
Abstract
The rapidly enlarging neural network models are becoming increasingly challenging to run on a single device. Hence model parallelism over multiple devices is critical to guarantee the efficiency of training large models. Recent proposals fall short either in long processing time or poor performance. Therefore, we propose Celeritas, a fast framework for optimizing device placement for large models. Celeritas employs a simple but efficient model parallelization strategy in the Standard Evaluation, and generates placement policies through a series of scheduling algorithms. We conduct experiments to deploy and evaluate Celeritas on numerous large models. The results show that Celeritas not only reduces the placement policy generation time by 26.4\% but also improves the model running time by 34.2\% compared to most advanced methods.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFerroelectric and Negative Capacitance Devices · IoT and Edge/Fog Computing · Advanced Neural Network Applications
