Assessing the trade-off between prediction accuracy and interpretability for topic modeling on energetic materials corpora

TL;DR

This paper investigates the balance between prediction accuracy and interpretability in topic modeling for energetics research, using various embedding methods and local explanations validated by domain experts.

Contribution

It introduces a comparative analysis of embedding methods and applies local interpretability techniques to energetics datasets, addressing domain-specific challenges.

Findings

Different embedding methods vary in accuracy and interpretability.

Local explanations help validate classifier decisions with domain experts.

The study provides insights into optimal trade-offs for energetics research.

Abstract

As the amount and variety of energetics research increases, machine aware topic identification is necessary to streamline future research pipelines. The makeup of an automatic topic identification process consists of creating document representations and performing classification. However, the implementation of these processes on energetics research imposes new challenges. Energetics datasets contain many scientific terms that are necessary to understand the context of a document but may require more complex document representations. Secondly, the predictions from classification must be understandable and trusted by the chemists within the pipeline. In this work, we study the trade-off between prediction accuracy and interpretability by implementing three document embedding methods that vary in computational complexity. With our accuracy results, we also introduce local interpretability model-agnostic explanations (LIME) of each prediction to provide a localized understanding of each prediction and to validate classifier decisions with our team of energetics experts. This study was carried out on a novel labeled energetics dataset created and validated by our team of energetics experts.