# Reliable evaluation for the AI-enabled intrusion detection system from data perspective

## TL;DR

This paper introduces a method to evaluate the reliability of AI-based intrusion detection systems by assessing data quality.

## Contribution

The novel contribution is a multi-indicator comprehensive evaluation method (MICEM) for ensuring reliable AI decisions in intrusion detection.

## Key findings

- The MICEM method evaluates intrusion detection data quality using multiple indicators.
- The method was validated on the CICIDS2017 dataset and a real intrusion detection dataset.
- The results show MICEM effectively ensures data usability and reliability for AI-enabled IDS.

## Abstract

As the primary link in cybersecurity, the intrusion detection system (IDS) is of indispensable importance. Many studies have proposed sophisticated artificial intelligence (AI) models to detect intrusion behavior from a large amount of data, yet they have ignored the fact that poor data quality has a direct impact on the performance of IDS. The poor data quality is mainly attributed to the interference and damage, such as data tampering, poisoning, and corruption, which leads to decision-making deviations, triggering a serious trust crisis of model application. This paper proposes a multi-indicator comprehensive evaluation method (MICEM) to ensure the reliability of AI decision-making from data perspective. First, several evaluation indicators are established to analyze the potential risks that intrusion detection data may face from the different dimensions, and specific quantitative methods are provided. Second, a comprehensive evaluation is conducted based on the results of each indicator to determine the quality of the intrusion detection data as a whole, thus guaranteeing the usability and reliability of AI-enabled IDS. Finally, the effectiveness and practicality of the proposed MICEM are fully verified by evaluating the benchmark-CICIDS2017 dataset and the real intrusion detection dataset.