CognitiveOS: Large Multimodal Model based System to Endow Any Type of Robot with Generative AI
Artem Lykov, Mikhail Konenkov, Koffivi Fid\`ele Gbagbe, Mikhail, Litvinov, Denis Davletshin, Aleksey Fedoseev, Miguel Altamirano Cabrera,, Robinroy Peter, Dzmitry Tsetserukou

TL;DR
CognitiveOS is a modular, transformer-based operating system for cognitive robots that enhances adaptability, scalability, and task understanding across diverse platforms, facilitating advanced real-world robotic applications.
Contribution
This work introduces the first modular operating system for cognitive robots, enabling seamless integration, configuration, and scalability of modules for diverse robotic tasks.
Findings
System outperforms CognitiveDog by 15% in reasoning tasks
Achieves 77% highest reasoning accuracy to date
Demonstrates adaptability across various robotic platforms
Abstract
This paper introduces CognitiveOS, the first operating system designed for cognitive robots capable of functioning across diverse robotic platforms. CognitiveOS is structured as a multi-agent system comprising modules built upon a transformer architecture, facilitating communication through an internal monologue format. These modules collectively empower the robot to tackle intricate real-world tasks. The paper delineates the operational principles of the system along with descriptions of its nine distinct modules. The modular design endows the system with distinctive advantages over traditional end-to-end methodologies, notably in terms of adaptability and scalability. The system's modules are configurable, modifiable, or deactivatable depending on the task requirements, while new modules can be seamlessly integrated. This system serves as a foundational resource for researchers and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAI-based Problem Solving and Planning · Robotics and Automated Systems
