ATLAS user analysis on private cloud resources at GoeGrid

Fabian Korte, Jordi Nadal Serrano, Jens Grabowski, Arnulf Quadt

Abstract

User analysis job demands can exceed available computing resources, especially before major conferences. ATLAS physics results might be slowed down due to this lack of resources available. For these reasons, cloud R&D activities are now included in the skeleton of the ATLAS computing model, which has been extended by using resources from commercial and private cloud providers to satisfy the demand. However, most of these activities are focused on Monte-Carlo production jobs, extending the resources at Tier-2. To evaluate the suitability of the cloud-computing model for user analysis jobs, we developed a framework to launch an ATLAS user analysis cluster in a cloud infrastructure on demand and evaluated two solutions. The first solution is totally integrated in the Grid infrastructure by using the same mechanism, which is already in use at Tier-2: A designated Panda-Queue is monitored and additional worker nodes are launched in a cloud environment and assigned to a corresponding HTCondor queue according to the demand. Thereby, the use of cloud resources is totally transparent to the user. However, using this approach, submitted user analysis jobs might still suffer from a certain delay introduced by waiting in the queue. Therefore, our second solution offers the possibility to easily deploy a totally private analysis cluster, i.e., batch or PROOF, on private cloud resources belonging to the university. Thereby, the private analysis cluster is connected to the ATLAS data-management system (DDM) to read and write input/output files.
Document Type: 
Articles in Conference Proceedings
Booktitle: 
Journal of Physics: Conference Series, Proceedings of the 21st Conference on Computing in High Energy Physics and Nuclear Physics (CHEP)
Language: 
English
Address: 
Okinawa, Japan
Publisher: 
IOP Publishing Ltd
Volume: 
664
Number: 
2
Pages: 
022020
Month: 
4
Year: 
2015
URL: 
http://iopscience.iop.org/article/10.1088/1742-6596/664/2/022020

Main menu 2

2011 © Software Engineering For Distributed Systems Group