Heterogeneous computing platform for real-time robotics
Jakub Fil, Yulia Sandamirskaya, Hector Gonzalez, Lo\"ic Azzalin, Stefan Gl\"uge, Lukas Friedenstab, Friedrich Wolf, Tim Rosmeisl, Matthias Lohrmann, Mahmoud Akl, Khaleel Khan, Leonie Wolf, Kristin Richter, Holm Puder, Mazhar Ali Bari, Xuan Choo, Noha Alharthi, Michael Hopkins

TL;DR
This paper proposes a heterogeneous computing platform combining neuromorphic hardware and traditional AI clusters to enable real-time, interactive robotics in smart city environments, demonstrated through a humanoid robot playing music with a human.
Contribution
It introduces a hybrid computing architecture integrating neuromorphic processors with GPU clusters for enhanced robotic perception and interaction in real-time applications.
Findings
Successful integration of Loihi2 neuromorphic hardware with GPU-based AI for robotics
Real-time performance in a humanoid robot playing music with a human
Demonstrated potential for heterogeneous architectures in complex robotic tasks
Abstract
After Industry 4.0 has embraced tight integration between machinery (OT), software (IT), and the Internet, creating a web of sensors, data, and algorithms in service of efficient and reliable production, a new concept of Society 5.0 is emerging, in which infrastructure of a city will be instrumented to increase reliability, efficiency, and safety. Robotics will play a pivotal role in enabling this vision that is pioneered by the NEOM initiative - a smart city, co-inhabited by humans and robots. In this paper we explore the computing platform that will be required to enable this vision. We show how we can combine neuromorphic computing hardware, exemplified by the Loihi2 processor used in conjunction with event-based cameras, for sensing and real-time perception and interaction with a local AI compute cluster (GPUs) for high-level language processing, cognition, and task planning. We…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Memory and Neural Computing · Ferroelectric and Negative Capacitance Devices · Modular Robots and Swarm Intelligence
