Optimised access to user analysis data using the gLite DPM

TL;DR

This paper discusses how ScotGrid's storage system, using gLite DPM middleware, was optimized for efficient user access to LHC data, analyzing performance and proposing improvements for end-user analysis workflows.

Contribution

The paper presents a design and performance analysis of ScotGrid's storage management to enhance user access to LHC data on the grid.

Findings

Optimized data access improves analysis throughput.

Procedures identified to reduce system load during analysis.

Performance analysis guides future system improvements.

Abstract

The ScotGrid distributed Tier-2 now provides more that 4MSI2K and 500TB for LHC computing, which is spread across three sites at Durham, Edinburgh and Glasgow. Tier-2 sites have a dual role to play in the computing models of the LHC VOs. Firstly, their CPU resources are used for the generation of Monte Carlo event data. Secondly, the end user analysis data is distributed across the grid to the site's storage system and held on disk ready for processing by physicists' analysis jobs. In this paper we show how we have designed the ScotGrid storage and data management resources in order to optimise access by physicists to LHC data. Within ScotGrid, all sites use the gLite DPM storage manager middleware. Using the EGEE grid to submit real ATLAS analysis code to process VO data stored on the ScotGrid sites, we present an analysis of the performance of the architecture at one site, and procedures that may be undertaken to improve such. The results will be presented from the point of view of the end user (in terms of number of events processed/second) and from the point of view of the site, which wishes to minimise load and the impact that analysis activity has on other users of the system.