A Stackelberg Model for Hybridization in Cryptography

TL;DR

This paper models the strategic interaction in cryptographic hybridization as a Stackelberg game, optimizing the defender's encryption choices and attacker's cryptanalysis under resource constraints.

Contribution

It introduces a novel game-theoretic framework for cryptographic hybridization, including a dynamic programming approach for the attacker and a linear program for the defender.

Findings

The model captures the strategic decision-making process of both defender and attacker.

A dynamic programming method is developed for the attacker's optimization.

The defender's problem is formulated as a linear program for optimal strategy selection.

Abstract

Similar to a strategic interaction between rational and intelligent agents, cryptography problems can be examined through the prism of game theory. In this setting, the agent aiming to protect a message is called the defender, while the one attempting to decrypt it, generally for malicious purposes, is the attacker. To strengthen security in cryptography, various strategies have been developed, among which hybridization stands out as a key concept in modern cryptographic design. This strategy allows the defender to select among different encryption algorithms (classical, post-quantum, or hybrid) while carefully balancing security and operational costs. On the other side, the attacker, limited by available resources, chooses cryptanalysis methods capable of breaching the selected algorithm. We model this interaction as a Stackelberg cryptographic hybridization problem under resource constraints. Here, the defender randomizes over encryption algorithms, and the attacker observes the choice before selecting suitable cryptanalysis methods. The attacker's decision is framed as a conditional optimization problem, which we refer to as the ``attacker subgame''. We then propose a dynamic programming approach for the attacker's subgame, while the defender's Stackelberg optimization is formulated as a linear program.