A solution for secure use of Kibana and Elasticsearch in multi-user environment

TL;DR

This paper presents a flexible, secure multi-user solution for Kibana and Elasticsearch, integrating user/group access control, Kerberos authentication, and Kibana plugins, enhancing data security and sharing capabilities.

Contribution

It introduces a novel, adaptable approach for user/group-based access control in Kibana and Elasticsearch, surpassing previous homemade plugins.

Findings

The solution effectively enforces data access restrictions in CC-IN2P3.

Integration with Search Guard maintains performance levels.

The approach is applicable to various environments beyond cloud services.

Abstract

Monitoring is indispensable to check status, activities, or resource usage of IT services. A combination of Kibana and Elasticsearch is used for monitoring in many places such as KEK, CC-IN2P3, CERN, and also non-HEP communities. Kibana provides a web interface for rich visualization, and Elasticsearch is a scalable distributed search engine. However, these tools do not support authentication and authorization features by default. In the case of single Kibana and Elasticsearch services shared among many users, any user who can access Kibana can retrieve other's information from Elasticsearch. In multi-user environment, in order to protect own data from others or share part of data among a group, fine-grained access control is necessary. The CERN cloud service group had provided cloud utilization dashboard to each user by Elasticsearch and Kibana. They had deployed a homemade Elasticsearch plugin to restrict data access based on a user authenticated by the CERN Single Sign On system. It enabled each user to have a separated Kibana dashboard for cloud usage, and the user could not access to other's one. Based on the solution, we propose an alternative one which enables user/group based Elasticsearch access control and Kibana objects separation. It is more flexible and can be applied to not only the cloud service but also the other various situations. We confirmed our solution works fine in CC-IN2P3. Moreover, a pre-production platform for CC-IN2P3 has been under construction. We will describe our solution for the secure use of Kibana and Elasticsearch including integration of Kerberos authentication, development of a Kibana plugin which allows Kibana objects to be separated based on user/group, and contribution to Search Guard which is an Elasticsearch plugin enabling user/group based access control. We will also describe the effect on performance from using Search Guard.