Catching Unusual Traffic Behavior using TF-IDF-based Port Access Statistics Analysis

TL;DR

This paper introduces a novel approach using TF-IDF, a natural language processing technique, to identify unusual network traffic behavior by analyzing port access logs, aiding in early detection of potential security threats.

Contribution

The study applies TF-IDF to network logs, mapping text analysis methods to port and access history, enabling detection of rare and potentially suspicious traffic patterns.

Findings

Successfully detected bot-oriented accesses

Identified unique UDP traffic patterns

Demonstrated effectiveness on real-world data

Abstract

Detecting the anomalous behavior of traffic is one of the important actions for network operators. In this study, we applied term frequency - inverse document frequency (TF-IDF), which is a popular method used in natural language processing, to detect unusual behavior from network access logs. We mapped the term and document concept to the port number and daily access history, respectively, and calculated the TF-IDF. With this approach, we could obtain ports frequently observed in fewer days compared to other port access activities. Such access behaviors are not always malicious activities; however, such information is a good indicator for starting a deeper analysis of traffic behavior. Using a real-life dataset, we could detect two bot-oriented accesses and one unique UDP traffic.