Learning to Fingerprint the Latent Structure in Question Articulation

TL;DR

This paper introduces a mathematical and neural auto-encoder based approach to recognize and refine the latent structure in question articulation, achieving high accuracy and nearly perfect recognition in experimental evaluations.

Contribution

It presents a novel objective-driven model for capturing latent question structures and demonstrates its effectiveness with neural auto-encoders and a refinement scheme called K-fingerprints.

Findings

80% recognition accuracy across question clusters

Nearly 100% recognition with K-fingerprints

Negligible false positives in experiments

Abstract

Abstract Machine understanding of questions is tightly related to recognition of articulation in the context of the computational capabilities of an underlying processing algorithm. In this paper a mathematical model to capture and distinguish the latent structure in the articulation of questions is presented. We propose an objective-driven approach to represent this latent structure and show that such an approach is beneficial when examples of complementary objectives are not available. We show that the latent structure can be represented as a system that maximizes a cost function related to the underlying objective. Further, we show that the optimization formulation can be approximated to building a memory of patterns represented as a trained neural auto-encoder. Experimental evaluation using many clusters of questions, each related to an objective, shows 80% recognition accuracy and negligible false positive across these clusters of questions. We then extend the same memory to a related task where the goal is to iteratively refine a dataset of questions based on the latent articulation. We also demonstrate a refinement scheme called K-fingerprints, that achieves nearly 100% recognition with negligible false positive across the different clusters of questions.