SoK: Reshaping Research on Network Intrusion Detection Systems

TL;DR

This paper critically examines the disconnect between research on Network Intrusion Detection Systems (NIDS) and real-world application, highlighting intrinsic challenges and proposing a foundation for future research improvements.

Contribution

It offers a reflective analysis of NIDS research, identifying fundamental misunderstandings and providing recommendations to align research with operational realities.

Findings

Many evaluations lack real network context

Compromised NIDS cannot be assumed effective

Triage by security operators differs from classifier outputs

Abstract

Network Intrusion Detection Systems (NIDS) have been studied for decades. Hundreds of papers have, e.g., proposed ways to enhance, harden or bypass NIDS. However, the findings of prior literature are hardly reflected in real-world operational contexts. Such a disconnection is problematic for research itself: it is unclear what scenario envisioned by prior work can be used as a baseline for future advancements. We argue that a key reason for this disconnection is a fundamental misunderstanding of intrinsic characteristics of NIDS. For instance, the fact that a compromised NIDS cannot be expected to work well; the fact that some evaluations are done without carrying out any experiment in a (even synthetic) "real" network; the fact that security operators triage high-level reports -- and not individual samples flagged by some classifier. In this SoK, which is primarily a reflective piece, we first constructively highlight such quintessential properties (without criticizing _any_ work by different authors) by stating three Assertions. Then, we provide recommendations -- further emphasized through an original and reproducible case study that challenges some established practices. Ultimately, we seek to lay a foundation to reshape research on NIDS.