MetaCentrum in e-Infrastructure era

Miroslav Ruda

CESNET, z.s.p.o., 160 00 Praha

In recent years, various infrastructure projects are being prepared on the European level (ESFRI projects) on national level (Research and Development for Innovations program) in different scientific areas. In IT area, three major infrastructure projects are emerging in Czech Republic (eIGeR, CERIT-SC and IT4Innovations) with common goal -- support IT-related requirements of projects from various scientific areasand create common e-Infrastructure, which will will encompass advance networks, computing and storage capacitiesand higher-level services. In this presentation, position of MetaCentrum will be presented, together with our actual and future plans, and services already provided or planned.

Under umbrella of CESNET e-Infrastructure project eIGeR, where grid computing is one of four major directions of development (together with national optical network, storage services and collaborative tools), MetaCentrum project is transforming itself to National Grid Infrastructure (NGI), which will cover not only provisioning of computational resources to users, but also interoperability with international grid infrastructures (namely EGI) and access coordination to these services with remaining IT providers.

In following years, moderate expansion of computational clusters already provided by MetaCentrum is expected, with first acquisition planned in summer 2011 (additional 500 CPU cores, 100TB of disk space) and similar extensions are in next years too. MetaCentrum resources will encompass cluster installations in several cities (Pilsen, Prague, Brno according current plans) with hardware architecture following currently deployed clusters (x86 architecture, mixture of high-density and larger shared-memory (SMP) nodes, accompanied with large shared filesystem). However, this resources should be perceived as a "glue", which should allow integration of larger resources, provided by other IT projects but also integration of clusters directly purchased by scientific projects from different application areas. To support integration of such more independent clusters, we have prepared major change in job management and scheduling tools, which includes transition from central scheduling system to decentralized pool of cooperating scheduling systems and further development in area of virtualization. We will present first results of this development, together with practical impact to users and owners of clusters already connected to MetaCentrum. We will also present first features inspired by cloud evolution, which allows easier adoption of this computing model in science.

National grid infrastructure must be directly connected to similar projects abroad. NGI will follow up with established cooperation with EGI, European project which build corresponding infrastructure across Europe. Native integration of national resources, or even resources (clusters) owned by research groups, into worldwide collaborative environment will support easier and seamless cooperation with scientific partners across the world. Transformation of MetaCentrum to more stable and dependable NGI gave us possibility to expand our services to areas, which are either induced from empowered coordination role or traditionally provided by standard high-performance centers. NGI will provide unified access to compute resources provided by various centers and will be, in cooperation with national and European partners, working on development, deployment and enforcement of these standards across all IT providers.

New possible area of cooperation is deployment of various application software tools; NGI itself will have a budget, which will allow broader set of development and application tools to be deployed in NGI, but we will also offer cooperation in software purchase process, again not only with other IT projects, but also with large scientific groups, which are also planning acquisition of new software tools and licenses in their projects.

Based of long experience with management of grid systems and with development of such environment, including non-trivial experience with evolving cloud paradigm, NGI will continue to provide modern computing environment not only for users, but also for scientific groups, which will own their own computational resources and will be willing to integrate them to larger infrastructure. NGI will also provide assistance to users and provide IT expertise, required for successful adoption of cluster or grid programming and optimization of computational code to such environments.