A User Interface for Sense-making of the Reasoning Process while Interacting with Robots
Chao Wang, Joerg Deigmoeller

TL;DR
This paper presents a user interface that visualizes robot reasoning processes by linking natural language commands, knowledge graph queries, and real-world 3D mapping to improve understanding and supervision of robotic actions.
Contribution
The authors developed an interactive interface that connects natural language commands with knowledge graph reasoning and real-world visualization, enhancing expert understanding of robot decision-making.
Findings
User feedback indicates improved understanding of robot reasoning.
System facilitates testing and modification of robot knowledge and actions.
Positive reception from robotic experts after 3 weeks of use.
Abstract
As knowledge graph has the potential to bridge the gap between commonsense knowledge and reasoning over actionable capabilities of mobile robotic platforms, incorporating knowledge graph into robotic system attracted increasing attention in recent years. Previously, graph visualization has been used wildly by developers to make sense of knowledge representations. However, due to lacking the link between abstract knowledge of the real-world environment and the robot's actions, transitional visualization tools are incompatible for expert-user to understand, test, supervise and modify the graph-based reasoning system with the embodiment of the robots. Therefore, we developed an interface which enables robotic experts to send commands to the robot in natural language, then interface visualizes the procedures of the robot mapping the command to the functions for querying in the commonsense…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSemantic Web and Ontologies · Robotics and Automated Systems · AI-based Problem Solving and Planning
