Scalable Deployment of AI Time-series Models for IoT

TL;DR

This paper introduces IBM Research Castor, a scalable cloud-native system for deploying and managing large numbers of AI time-series models in IoT, supporting model reuse, automation, and traceability.

Contribution

It presents a novel system architecture that enables scalable deployment, automated management, and traceability of AI time-series models in IoT environments.

Findings

Successfully deployed in real-world smart-grid applications

Supported tens of thousands of AI modeling tasks

Demonstrated scalability and automation in cloud environment

Abstract

IBM Research Castor, a cloud-native system for managing and deploying large numbers of AI time-series models in IoT applications, is described. Modelling code templates, in Python and R, following a typical machine-learning workflow are supported. A knowledge-based approach to managing model and time-series data allows the use of general semantic concepts for expressing feature engineering tasks. Model templates can be programmatically deployed against specific instances of semantic concepts, thus supporting model reuse and automated replication as the IoT application grows. Deployed models are automatically executed in parallel leveraging a serverless cloud computing framework. The complete history of trained model versions and rolling-horizon predictions is persisted, thus enabling full model lineage and traceability. Results from deployments in real-world smart-grid live forecasting applications are reported. Scalability of executing up to tens of thousands of AI modelling tasks is also evaluated.