The Vacuity of the Open Source Security Testing Methodology Manual

TL;DR

This paper critically examines the OSSTMM, revealing that its core principles are flawed, its security scoring is meaningless, and its approach to human security is problematic, ultimately arguing it should be discarded.

Contribution

It provides a fundamental textual critique of OSSTMM, exposing its conceptual flaws and arguing against its continued use in security testing.

Findings

OSSTMM's security score is an empty abstraction.

The trust metric in OSSTMM is confusing and meaningless.

OSSTMM's view of human security is overly threat-focused.

Abstract

The Open Source Security Testing Methodology Manual (OSSTMM) provides a "scientific methodology for the accurate characterization of operational security" [Her10, p.13]. It is extensively referenced in writings aimed at security testing professionals such as textbooks, standards and academic papers. In this work we offer a fundamental critique of OSSTMM and argue that it fails to deliver on its promise of actual security. Our contribution is threefold and builds on a textual critique of this methodology. First, OSSTMM's central principle is that security can be understood as a quantity of which an entity has more or less. We show why this is wrong and how OSSTMM's unified security score, the rav, is an empty abstraction. Second, OSSTMM disregards risk by replacing it with a trust metric which confuses multiple definitions of trust and, as a result, produces a meaningless score. Finally, OSSTMM has been hailed for its attention to human security. Yet it understands all human agency as a security threat that needs to be constantly monitored and controlled. Thus, we argue that OSSTMM is neither fit for purpose nor can it be salvaged, and it should be abandoned by security professionals.