• Newsfeed
• Newsletters
• Presentations
• Publications
• Director's letters
• Press

The EGI Operations Portal

Cyril L’Orphelin introduces the latest improvements

The EGI Operations Portal is the official entry point for all information and services related to EGI’s operations, where the community can manage, monitor, share and discuss information.

As of July 2011, the portal is divided into two instances, both developed and hosted in the IN2P3 Computing Center in Lyon, France:

• The historical CIC Portal

• The new Operations Portal based on the Symfony framework 

Operations Portal: The story so far

When EGEE-I started, the infrastructure was smaller and was managed centrally from the Operations Centre at CERN. While this worked quite well, troubleshooting of fifty sites was hard because grid operations expertise was concentrated in one place. From October 2004, several federations of countries started to share shifts to spread out the expertise and the responsibility for managing the infrastructure around the globe on a weekly basis. Indeed, this reduced the workload.

However, requirements on tools synchronisation and communication soared along with the complexity of the work. It became necessary to have all operational tools available through a single interface to allow an inter-active and integrated use of these tools. This inspired the dashboard for Central Operators on Duty (COD dashboard), which became one of the main features in the first operations portal. The need of a management and operations tool for EGEE and WLCG (Worldwide LCG) lead, in 2004, to the creation of the EGEE Operations Portal, later referred to as ‘the CIC Portal’, to provide an entry point for all EGEE actors for their operational needs.

The historical CIC Portal has been built as an integration platform, allowing for strong interaction among existing tools with similar scope but also filling up gaps wherever functionality has been lacking. Many of the workflows came out of requirements expressed by end-users or administrators of Virtual Organisations (VO), Regional Operations Centres (ROCs) or Resource Centres and the Operations Coordination Centre of EGEE. Following the same principle, the Operations Portal has been developed from an ‘actor’s view’, where each member of the community has access to information according to his role in the project. This includes the grid operator who monitors resources and grid services daily, regular grid users, as well as VO, site or National Grid Infrastructure (NGI) managers. The portal also fosters communication between different actors, through channels such as broadcast or downtime notifications, and has set-up procedures to address their interaction needs. top

Architecture: A look inside

The architecture is the same for both versions of the portal and is made of three modules:

• A database – to store information related to the users or the VO

• A web module – graphical user interface – which is currently integrated into the Symfony framework

• A Data Aggregation and Unification Service named Lavoisier

Lavoisier is the component used to store, consolidate and ‘feed’ data into the web application. It provides information from various sources, which protects the application from intermittent failures of information sources.

The application was developed to enable easy and efficient cross-data sources queries, independently of technologies used. Data views are represented as XML documents and the query language is XSL. top

Year 1 accomplishments: Migration to the new portal

One of the biggest achievements of the first year of EGI activity was the successful migration from the historical CIC Portal to the new Operations Portal, which involved a significant amount of effort.

The improvements were developed and integrated within a new framework named Symfony. The benefits we have seen so far are three-fold:

• An increase in efficiency of the Operations Portal application in terms of response time to end-user requests.

• A decrease in time spent on software maintenance.

• An increase in robustness of the application by developing independent modules.

The Symfony framework approach improves the organisation of the source code, which leads to a high-rate of re-usability. It also brings additional features such as a security layer assuming XSS and CSRF protection, a set of different work environments and the use of plugins developed by the Symfony community.

Features migrated into the new framework

The VO ID card. This system records the life cycle of a given VO and links the VO mana-gers to the project management for operations. The data is stored in the CIC DB, hosted at CC-IN2P3. The VO ID cards are the static repository for VOs with information such:

• the VO contact points (e.g. mana-gers, user mailing list, representatives);

• the VO global information (e.g. enrolment URL, status, discipline, Acceptable User Policy, the specific requirements);

• the VO Membership Service (VOMS) information (e.g. groups and roles, certificate details)

The Dashboard. Allows operations staff to track problems using different results from the various Monitoring Tools (SAM, Nagios and Gstat) and to open or update trouble tickets. The Dashboard uses the GOCDB to consolidate moni-toring information with downtime information, and GSTAT to provide dynamic statuses (Storage Usage, CPU Usage, number of jobs waiting and running).

The broadcast tool. With this tool, every user authenti-cated with a grid certificate is able to contact several categories of stake-holders interested in a problem, an announcement or in a specific release. The aim is to provide everyone in the grid community with information by mail or RSS feeds. The current model is based on information from GOCDB (sites or ROC/NGI contacts) and from the CIC DB (VO contacts, mailing lists for operations). In addition to sending out information, the broadcast tools also provide an archiving service and a search engine. top

Year 1 accomplishments: The regional package

The portal team has also developed a regional package to distribute the different features and modules of the Central Operations Portal in a format that allows NGIs to adapt the portal to their individual needs.

The first regional package was released on 8 June 2010 and included the first release of the dashboard module. The different modules of the package and related documentation are distributed via a SVN repository.

The application is composed of a web service Lavoisier and a PHP web application to provide a user interface. The database is generated automatically during the first installation.

The regional portal is linked with the central instance of Lavoisier; creation, update, delete of records are synchronised so as not to disrupt global oversight operations.

The central and the regional instances were built on the same model to behave in the same way and to be easily interoperable.

Synchronisation between the regional portals and the central instance is achieved through REST and SOAP. Records are synchronised every five minutes using php scripts. Any problem detected during the synchronisation is reported in a mail sent to webmasters.

The regional package is currently deployed in four National Grid Initiatives:

• the Czech NGI

• the Greek NGI

• the Ibergrid NGI

• the Belarus NGI top