TL;DR
MLModelCI is an open-source cloud platform that automates the optimization, profiling, containerization, and deployment of machine learning models, streamlining MLaaS development and management.
Contribution
It introduces an automated pipeline for converting, profiling, and deploying ML models with elastic evaluation, bridging training and serving systems.
Findings
Automates model optimization and deployment process.
Provides profiling data for performance-cost trade-offs.
Supports elastic evaluation to maintain service quality.
Abstract
MLModelCI provides multimedia researchers and developers with a one-stop platform for efficient machine learning (ML) services. The system leverages DevOps techniques to optimize, test, and manage models. It also containerizes and deploys these optimized and validated models as cloud services (MLaaS). In its essence, MLModelCI serves as a housekeeper to help users publish models. The models are first automatically converted to optimized formats for production purpose and then profiled under different settings (e.g., batch size and hardware). The profiling information can be used as guidelines for balancing the trade-off between performance and cost of MLaaS. Finally, the system dockerizes the models for ease of deployment to cloud environments. A key feature of MLModelCI is the implementation of a controller, which allows elastic evaluation which only utilizes idle workers while…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
