New Hashing Algorithm for Use in TCP Reassembly Module of IPS

TL;DR

This paper introduces a novel cryptography-based hashing algorithm designed to improve TCP connection management in IDS/IPS systems by providing more efficient and evenly distributed hash keys for reassembly modules.

Contribution

The paper proposes a new cryptography-inspired hashing algorithm that enhances TCP reassembly in IDS/IPS by using the full four-tuple for better connection identification.

Findings

Improved hash key distribution over traditional methods

Enhanced efficiency in TCP reassembly process

Better detection of distributed malicious code

Abstract

Since last decade, IDS/ IPS has gained popularity in protecting large networks. They can employ signature based techniques and/or flow-based techniques to prevent intrusion from outside/ inside the network they are trying to protect. Signature based IDS/ IPS can be stateless or stateful. Stateful IDS can store the state of the protocol and use it for better detection of malware. In the case of TCP/IP networks, an attacker can also launch an attack such that the malicious code is distributed over many packets. These packets pass through the traditional IDS/ IPS and reassemble inside the network. Once re-assembled inside the network by the TCP/IP layer, the malicious code launches an attack. The TCP state and a copy of last few packets for each active connection has to be maintained in IDS/IPS. In TCP re-assembly, packets are re-assembled at IDS/IPS and searched for signature matches. A connection table has to be maintained for active connections and their list of last few (atmost 11) packets that have already arrived. We need data structures for searching the connection that the latest incoming packet belongs to. Popular hashing algorithms like CRC, XOR, summing tuple, taking modulus are inefficient as hash keys are not evenly distributed in hash-key space. Thus we show how an algorithm based on cryptography concepts can be used for efficient hashing in network connection management. We also show how to use full four tuple for calculating hash key instead of simply summing the tuple and taking the modulus of the sum.