Boosting AI Reliability with an FSM-Driven Streaming Inference Pipeline: An Industrial Case

TL;DR

This paper introduces an FSM-driven streaming inference pipeline that improves AI robustness in industrial applications by integrating prior knowledge, demonstrated through an excavator workload counting system with enhanced accuracy and reliability.

Contribution

The paper presents a novel streaming inference pipeline combining object detection with an FSM to incorporate prior knowledge, improving robustness in industrial AI tasks.

Findings

Superior performance over heuristic-based methods

Enhanced robustness in real-world scenarios

Effective integration of FSM with streaming inference

Abstract

The widespread adoption of AI in industry is often hampered by its limited robustness when faced with scenarios absent from training data, leading to prediction bias and vulnerabilities. To address this, we propose a novel streaming inference pipeline that enhances data-driven models by explicitly incorporating prior knowledge. This paper presents the work on an industrial AI application that automatically counts excavator workloads from surveillance videos. Our approach integrates an object detection model with a Finite State Machine (FSM), which encodes knowledge of operational scenarios to guide and correct the AI's predictions on streaming data. In experiments on a real-world dataset of over 7,000 images from 12 site videos, encompassing more than 300 excavator workloads, our method demonstrates superior performance and greater robustness compared to the original solution based on manual heuristic rules. We will release the code at https://github.com/thulab/video-streamling-inference-pipeline.