Combining multimodal fatigue fracture surface images for analysis with a CNN

TL;DR

This paper explores using combined imaging techniques with a CNN to better analyze fatigue fractures in titanium alloy surfaces.

Contribution

The novel approach combines SEM, BSE, and SWLI images as color channels for CNNs to improve fracture surface analysis.

Findings

Combining imaging modalities improves classification accuracy by 20%.

Regression results improve by 60% when using multiple modalities.

Each modality contributes differently depending on the model.

Abstract

This work uses three different modalities, namely SEM, BSE and scanning white light interference (SWLI) to image fatigue fracture surfaces of Ti-6Al-4V. Convolutional neural networks (CNNs) that were pre-trained on images of the natural world were used to predict values such as distance from load line and crack growth rate. SEM images are routinely used to study the topography of fracture surfaces because the shallower interaction volume resolves surface features while BSE images and SWLI data add information about composition and surface height. Combining the three imaging modalities via the use of color channels facilitates overlaying them on the same grid for transferability of models pre-trained on colored images. This work shows that the imaging modalities under the guise of color channels have different levels of importance depending on the model being trained. It also documents how the combination of information from these modalities improves the classification and regression results by 20 and 60 %, respectively, relative to the secondary electron images alone. (162/200).

Figures