Towards VEsNA, a Framework for Managing Virtual Environments via Natural Language Agents

TL;DR

VEsNA is a framework that uses natural language processing and intelligent agents to simplify the creation and management of virtual environments, aiding factory automation design without requiring advanced programming skills.

Contribution

It introduces a novel framework combining agent-based technology and NLP for intuitive virtual environment management and reasoning in factory automation scenarios.

Findings

Prototype implementation of VEsNA demonstrates feasibility.

Natural language interface enables non-experts to manipulate virtual environments.

Cognitive reasoning supports compliance and what-if analysis.

Abstract

Automating a factory where robots are involved is neither trivial nor cheap. Engineering the factory automation process in such a way that return of interest is maximized and risk for workers and equipment is minimized, is hence of paramount importance. Simulation can be a game changer in this scenario but requires advanced programming skills that domain experts and industrial designers might not have. In this paper we present the preliminary design and implementation of a general-purpose framework for creating and exploiting Virtual Environments via Natural language Agents (VEsNA). VEsNA takes advantage of agent-based technologies and natural language processing to enhance the design of virtual environments. The natural language input provided to VEsNA is understood by a chatbot and passed to a cognitive intelligent agent that implements the logic behind displacing objects in the virtual environment. In the VEsNA vision, the intelligent agent will be able to reason on this displacement and on its compliance to legal and normative constraints. It will also be able to implement what-if analysis and case-based reasoning. Objects populating the virtual environment will include active objects and will populate a dynamic simulation whose outcomes will be interpreted by the cognitive agent; explanations and suggestions will be passed back to the user by the chatbot.