Knowledge mining of unstructured information: application to cyber-domain

TL;DR

This paper introduces a novel machine learning framework that constructs knowledge graphs from unstructured cyber incident reports, enabling more effective threat analysis and risk estimation in the cyber domain.

Contribution

The paper presents a new knowledge mining framework with a cyber-ontology and graph-based threat estimation, improving information extraction from unstructured cyber incident texts.

Findings

Knowledge extraction accuracy is sufficient for practical use.

Graph-based threat estimation correlates with actual attack records.

Framework helps analysts infer cyber risk propagation.

Abstract

Information on cyber-related crimes, incidents, and conflicts is abundantly available in numerous open online sources. However, processing the large volumes and streams of data is a challenging task for the analysts and experts, and entails the need for newer methods and techniques. In this article we present and implement a novel knowledge graph and knowledge mining framework for extracting the relevant information from free-form text about incidents in the cyberdomain. The framework includes a machine learning based pipeline for generating graphs of organizations, countries, industries, products and attackers with a non-technical cyber-ontology. The extracted knowledge graph is utilized to estimate the incidence of cyberattacks on a given graph configuration. We use publicly available collections of real cyber-incident reports to test the efficacy of our methods. The knowledge extraction is found to be sufficiently accurate, and the graph-based threat estimation demonstrates a level of correlation with the actual records of attacks. In practical use, an analyst utilizing the presented framework can infer additional information from the current cyber-landscape in terms of risk to various entities and propagation of the risk heuristic between industries and countries.