Automated Training and Maintenance through Kinect

TL;DR

This paper presents a low-cost, Kinect-based augmented reality system that simplifies automobile maintenance by enabling gesture and speech control, reducing the need for expert mechanics.

Contribution

The paper introduces a novel system combining Kinect's gesture recognition with augmented reality for automated vehicle maintenance assistance.

Findings

High accuracy gesture recognition with Kinect

Real-time AR overlay for maintenance guidance

Cost-effective system design

Abstract

In this paper, we have worked on reducing burden on mechanic involving complex automobile maintenance activities that are performed in centralised workshops. We have presented a system prototype that combines Augmented Reality with Kinect. With the use of Kinect, very high quality sensors are available at considerably low costs, thus reducing overall expenditure for system design. The system can be operated either in Speech mode or in Gesture mode. The system can be controlled by various audio commands if user opts for Speech mode. The same controlling can also be done by using a set of Gestures in Gesture mode. Gesture recognition is the task performed by Kinect system. This system, bundled with RGB and Depth camera, processes the skeletal data by keeping track of 20 different body joints. Recognizing Gestures is done by verifying user movements and checking them against predefined condition. Augmented Reality module captures real-time image data streams from high resolution camera. This module then generates 3D model that is superimposed on real time data.