Interactive Discovery of Coordinated Relationship Chains with Maximum Entropy Models

TL;DR

This paper introduces maximum entropy models integrated into a visual analytic system to help users discover surprising, coordinated relationship chains in large text datasets, aiding exploratory analysis in intelligence and security domains.

Contribution

The paper presents a novel maximum entropy modeling approach and a visual system, MERCER, for interactive discovery of relationship chains, enhancing human-in-the-loop analysis in complex datasets.

Findings

Effective identification of relationship chains in synthetic datasets.

Successful application to real intelligence datasets.

User input improves analysis accuracy.

Abstract

Modern visual analytic tools promote human-in-the-loop analysis but are limited in their ability to direct the user toward interesting and promising directions of study. This problem is especially acute when the analysis task is exploratory in nature, e.g., the discovery of potentially coordinated relationships in massive text datasets. Such tasks are very common in domains like intelligence analysis and security forensics where the goal is to uncover surprising coalitions bridging multiple types of relations. We introduce new maximum entropy models to discover surprising chains of relationships leveraging count data about entity occurrences in documents. These models are embedded in a visual analytic system called MERCER that treats relationship bundles as first class objects and directs the user toward promising lines of inquiry. We demonstrate how user input can judiciously direct analysis toward valid conclusions whereas a purely algorithmic approach could be led astray. Experimental results on both synthetic and real datasets from the intelligence community are presented.