Notebook and Workload manager services are two popular EGI services used by many scientific communities. Notebook provides an interactive data analysis interface relying on EGI Cloud and storage. On the other hand, when computation becomes large-scale, the DIRAC-based Workload Manager becomes useful for load balancing and maximising the usage of computational resources.
New requirements were raised by EISCAT Competence Center when they integrated both services to build a data access portal; in particular, they wanted the Notebook to directly access the DIRAC client toolkits, allowing EISCAT users to retrieve data from DIRAC storage and manage job submission without leaving the Notebook environment.
The identified solution foresees using CVMFS to provide DIRAC client components via POSIX interface. The integration solution has been tested by EISCAT CC in their newly released EISCAT Data Access Portal.
Today, we are pleased to announce this integration of Notebook with DIRAC-based Workload manager is in production. As it is generic, any community with similar requirements can consider its adoption.
This was a collaborative effort involving EGI Technical Solutions, CESNET and DIRAC teams. We would also give special thanks to EISCAT CC who lead the testing – all these valuable community requirements help advance the technology evolution in EGI.
