Using AI/ML to gain situational understanding from passive network observations

TL;DR

This paper presents an AI/ML-based system that analyzes passive network traffic in government buildings to enhance situational awareness by identifying devices, detecting policy violations, and uncovering security vulnerabilities.

Contribution

The paper introduces a novel system combining network domain knowledge and machine learning to extract actionable insights from passive network observations.

Findings

Effective device characterization and discovery

Detection of unauthorized and hidden devices

Identification of security vulnerabilities and information leaks

Abstract

The data available in the network traffic fromany Government building contains a significant amount ofinformation. An analysis of the traffic can yield insightsand situational understanding about what is happening inthe building. However, the use of traditional network packet inspection, either deep or shallow, is useful for only a limited understanding of the environment, with applicability limited to some aspects of network and security management. If weuse AI/ML based techniques to understand the network traffic, we can gain significant insights which increase our situational awareness of what is happening in the environment.At IBM, we have created a system which uses a combination of network domain knowledge and machine learning techniques to convert network traffic into actionable insights about the on premise environment. These insights include characterization of the communicating devices, discovering unauthorized devices that may violate policy requirements, identifying hidden components and vulnerability points, detecting leakage of sensitive information, and identifying the presence of people and devices.In this paper, we will describe the overall design of this system, the major use-cases that have been identified for it, and the lessons learnt when deploying this system for some of those use-cases