Balancing Interpretability and Predictive Accuracy for Unsupervised Tensor Mining

TL;DR

This paper investigates balancing interpretability and predictive accuracy in unsupervised tensor mining, specifically focusing on improving the estimation of the number of latent factors in PARAFAC decomposition.

Contribution

It explores the trade-off between interpretability and accuracy to enhance the automatic rank estimation method using CORCONDIA.

Findings

Balancing interpretability and accuracy improves rank estimation.

Preliminary results show benefits of trade-off in tensor decomposition.

Enhanced method leads to more reliable multi-aspect data mining.

Abstract

The PARAFAC tensor decomposition has enjoyed an increasing success in exploratory multi-aspect data mining scenarios. A major challenge remains the estimation of the number of latent factors (i.e., the rank) of the decomposition, which yields high-quality, interpretable results. Previously, we have proposed an automated tensor mining method which leverages a well-known quality heuristic from the field of Chemometrics, the Core Consistency Diagnostic (CORCONDIA), in order to automatically determine the rank for the PARAFAC decomposition. In this work we set out to explore the trade-off between 1) the interpretability/quality of the results (as expressed by CORCONDIA), and 2) the predictive accuracy of the results, in order to further improve the rank estimation quality. Our preliminary results indicate that striking a good balance in that trade-off benefits rank estimation.