Towards Near Real-Time BGP Deep Analysis: A Big-Data Approach

TL;DR

This paper introduces a scalable big-data platform for near real-time analysis of BGP data, enabling better detection and understanding of network anomalies and security incidents.

Contribution

It presents a novel Apache Spark-based application integrated with PNDA for real-time BGP data processing and analysis.

Findings

Enables near real-time detection of BGP anomalies

Supports scalable ingestion of live BGP feeds from multiple vantage points

Provides high-level insights into network security and stability

Abstract

BGP (Border Gateway Protocol) serves as the primary routing protocol for the Internet, enabling Autonomous Systems (individual network operators) to exchange network reachability information. Alongside significant on-going research and development efforts, there is a practical need to understand the nature of events that occur on the Internet. Network operators are acutely aware of security-related incidents such as 'Prefix Hijacking' as well as the impact of network instabilities that ripple through the Internet. Recent research focused on the study of BGP anomalies (both network/prefix instability and security-related incidents) has been based on the analysis of historical logs. Further analysis to understand the nature of these anomalous events is not always sufficient to be able to differentiate malicious activities, such as prefix- or sub-prefix- hijacking, from those events caused by inadvertent misconfigurations. In addition, such techniques are challenged by a lack of sufficient resources to store and process data feeds in real-time from multiple BGP Vantage Points (VPs). In this paper, we present a BGP Deep-analysis application developed using the PNDA (Platform for Network Data Analytics) 'Big-Data' platform. PNDA provides a highly scalable environment that enables the ingestion and processing of 'live' BGP feeds from many vantage points in a schema-agnostic manner. The Apache Spark-based application, in conjunction with PNDA's distributed processing capabilities, is able to perform high-level insights as well as near-to-real-time statistical analysis