The Living Application: a Self-Organising System for Complex Grid Tasks

TL;DR

This paper introduces the concept of a living application that autonomously manages and adapts its resource usage on the grid in real-time, demonstrated through a galaxy collision simulation switching between specialized computing resources.

Contribution

It presents the novel idea of a self-organising, autonomous application that dynamically adapts to resource topology changes during execution, reducing reliance on external workflow tools.

Findings

Successfully demonstrated real-time resource switching in a galaxy collision simulation.

Showed the application can adapt to different computational resources without user intervention.

Validated the concept with a test setup using GPU and GRAPE-enabled machines.

Abstract

We present the living application, a method to autonomously manage applications on the grid. During its execution on the grid, the living application makes choices on the resources to use in order to complete its tasks. These choices can be based on the internal state, or on autonomously acquired knowledge from external sensors. By giving limited user capabilities to a living application, the living application is able to port itself from one resource topology to another. The application performs these actions at run-time without depending on users or external workflow tools. We demonstrate this new concept in a special case of a living application: the living simulation. Today, many simulations require a wide range of numerical solvers and run most efficiently if specialized nodes are matched to the solvers. The idea of the living simulation is that it decides itself which grid machines to use based on the numerical solver currently in use. In this paper we apply the living simulation to modelling the collision between two galaxies in a test setup with two specialized computers. This simulation switces at run-time between a GPU-enabled computer in the Netherlands and a GRAPE-enabled machine that resides in the United States, using an oct-tree N-body code whenever it runs in the Netherlands and a direct N-body solver in the United States.