Platform for Situated Intelligence

TL;DR

Platform for Situated Intelligence is an open-source framework designed to facilitate rapid development, integration, and deployment of multimodal AI systems with tools for sensing, fusion, inference, visualization, and debugging.

Contribution

It introduces a comprehensive infrastructure and ecosystem for building multimodal, integrative-AI systems efficiently and effectively in real-world environments.

Findings

Supports rapid prototyping and development of multimodal AI systems

Provides tools for visualization and debugging of data streams

Ensures performance suitable for open-world deployment

Abstract

We introduce Platform for Situated Intelligence, an open-source framework created to support the rapid development and study of multimodal, integrative-AI systems. The framework provides infrastructure for sensing, fusing, and making inferences from temporal streams of data across different modalities, a set of tools that enable visualization and debugging, and an ecosystem of components that encapsulate a variety of perception and processing technologies. These assets jointly provide the means for rapidly constructing and refining multimodal, integrative-AI systems, while retaining the efficiency and performance characteristics required for deployment in open-world settings.