Implementation of New Security Features in CMSWEB Kubernetes Cluster at CERN

TL;DR

This paper details the implementation of advanced security features in the CERN CMSWEB Kubernetes cluster, including network policies, Open Policy Agent, and Vault, to enhance security and operational reliability.

Contribution

It introduces a comprehensive security infrastructure for Kubernetes clusters in a high-energy physics environment, integrating network policies, OPA, and Vault without modifying core Kubernetes components.

Findings

Enhanced security through network policies and OPA enforcement.

Secure, centralized management of sensitive data with Vault.

Improved reliability and security of CMSWEB services.

Abstract

The CMSWEB cluster is pivotal to the activities of the Compact Muon Solenoid (CMS) experiment, as it hosts critical services required for the operational needs of the CMS experiment. The security of these services and the corresponding data is crucial to CMS. Any malicious attack can compromise the availability of our services. Therefore, it is important to construct a robust security infrastructure. In this work, we discuss new security features introduced to the CMSWEB Kubernetes ("k8s") cluster. The new features include the implementation of network policies, deployment of Open Policy Agent (OPA), enforcement of OPA policies, and the integration of Vault. The network policies act as an inside-the-cluster firewall to limit the network communication between the pods to the minimum necessary, and its dynamic nature allows us to work with microservices. The OPA validates the objects against some custom-defined policies during create, update, and delete operations to further enhance security. Without recompiling or changing the configuration of the Kubernetes API server, it can apply customized policies on Kubernetes objects and their audit functionality enabling us to detect pre-existing conflicts and issues. Although Kubernetes incorporates the concepts of secrets, they are only base64 encoded and are not dynamically configured. This is where Vault comes into play: Vault dynamically secures, stores, and tightly controls access to sensitive data. This way, the secret information is encrypted, secured, and centralized, making it more scalable and easier to manage. Thus, the implementation of these three security features corroborate the enhanced security and reliability of the CMSWEB Kubernetes infrastructure.