Applying Deep Neural Networks to automate visual verification of manual bracket installations in aerospace

TL;DR

This paper presents a deep learning approach using Siamese Neural Networks for automated visual verification of manual aerospace bracket installations, demonstrating improved performance with novel voting schemes and transfer learning.

Contribution

It introduces a novel similarity voting scheme and applies deep neural networks to aerospace inspection, a first in this specific application domain.

Findings

Siamese Neural Networks perform well with limited training data.

The novel voting scheme enhances model accuracy.

Transfer learning improves verification performance.

Abstract

In this work, we explore a deep learning based automated visual inspection and verification algorithm, based on the Siamese Neural Network architecture. Consideration is also given to how the input pairs of images can affect the performance of the Siamese Neural Network. The Siamese Neural Network was explored alongside Convolutional Neural Networks. In addition to investigating these model architectures, additional methods are explored including transfer learning and ensemble methods, with the aim of improving model performance. We develop a novel voting scheme specific to the Siamese Neural Network which sees a single model vote on multiple reference images. This differs from the typical ensemble approach of multiple models voting on the same data sample. The results obtained show great potential for the use of the Siamese Neural Network for automated visual inspection and verification tasks when there is a scarcity of training data available. The additional methods applied, including the novel similarity voting, are also seen to significantly improve the performance of the model. We apply the publicly available omniglot dataset to validate our approach. According to our knowledge, this is the first time a detailed study of this sort has been carried out in the automatic verification of installed brackets in the aerospace sector via Deep Neural Networks.