Jarvis for Aeroengine Analytics: A Speech Enhanced Virtual Reality Demonstrator Based on Mining Knowledge Databases

TL;DR

This paper introduces a VR environment for aeroengine analysis that integrates speech and gesture interaction with knowledge-based systems, enabling engineers to query and analyze engine data in an immersive setting.

Contribution

It presents a novel VR demonstrator that combines CAD models, characteristic plots, speech and gesture inputs, and a knowledge-graph system for real-time aeroengine health analysis.

Findings

Enhanced interaction with engine data in VR environment

Knowledge-graph system enables complex queries and analysis

Progress towards a Jarvis-like aeroengine analytics system

Abstract

In this paper, we present a Virtual Reality (VR) based environment where the engineer interacts with incoming data from a fleet of aeroengines. This data takes the form of 3D computer-aided design (CAD) engine models coupled with characteristic plots for the subsystems of each engine. Both the plots and models can be interacted with and manipulated using speech or gestural input. The characteristic data is ported to a knowledge-based system underpinned by a knowledge-graph storing complex domain knowledge. This permits the system to respond to queries about the current state and health of each aeroengine asset. Responses to these questions require some degree of analysis, which is handled by a semantic knowledge representation layer managing information on aeroengine subsystems. This paper represents a significant step forward for aeroengine analysis in a bespoke VR environment and brings us a step closer to a Jarvis-like system for aeroengine analytics.