Comparative Study of Deep Learning Architectures for Textual Damage Level Classification

TL;DR

This study evaluates deep learning models like LSTM, BLSTM, GRU, and sRNN for classifying aircraft damage levels from unstructured incident narratives, achieving over 88% accuracy and highlighting NLP's potential in aviation safety analysis.

Contribution

It introduces a comparative analysis of multiple deep learning architectures for classifying damage levels from textual safety reports, demonstrating their effectiveness in aviation safety.

Findings

All models achieved over 88% accuracy.

sRNN was the top performer with 89% accuracy.

Deep learning models significantly outperform random guessing.

Abstract

Given the paramount importance of safety in the aviation industry, even minor operational anomalies can have significant consequences. Comprehensive documentation of incidents and accidents serves to identify root causes and propose safety measures. However, the unstructured nature of incident event narratives poses a challenge for computer systems to interpret. Our study aimed to leverage Natural Language Processing (NLP) and deep learning models to analyze these narratives and classify the aircraft damage level incurred during safety occurrences. Through the implementation of LSTM, BLSTM, GRU, and sRNN deep learning models, our research yielded promising results, with all models showcasing competitive performance, achieving an accuracy of over 88% significantly surpassing the 25% random guess threshold for a four-class classification problem. Notably, the sRNN model emerged as the top performer in terms of recall and accuracy, boasting a remarkable 89%. These findings underscore the potential of NLP and deep learning models in extracting actionable insights from unstructured text narratives, particularly in evaluating the extent of aircraft damage within the realm of aviation safety occurrences.