Porting and optimisation of the Met Office Unified Model on Petascale architectures
We present porting, optimisation and scaling results from our work with the United Kingdom's Unified Model on a number of massively parallel architectures: the UK MONSooN and HECToR systems, the German HERMIT and the French Curie supercomputer, part of the Partnership for Advanced Computing in Europe (PRACE). The model code used for this project is a configuration of the Met Office Unified Model (MetUM) called Global Atmosphere GA3.0, in its climate mode (HadGEM3, Walters et al., 2011, and Malcolm et al., 2010). The atmospheric dynamical core uses a semi-implicit, semi-Lagrangian scheme. The model grid is spherical (a lat/lon grid) and polar filtering is applied around the two singularities. For the configuration used on PRACE, with a horizontal grid spacing of 25km (N512) and 85 vertical levels up to 85km, we use a 10-minute time step. Initial conditions are derived from fully balanced coupled experiments at lower resolution and atmosphere/land surface perturbations are imposed using standard Met Office tools for ensemble initialisation. Initial development occurred on a NERC-MO joint facility, MONSooN, with 29 IBM-P6 nodes, using up to 12 nodes. In parallel with this activity, we have tested the model on the NERC/EPSRC supercomputer, HECToR (CRAY XE6), using 1'536 to 24'576 cores. The scaling breakthroughs came after implementing the use of hybrid parallelism: OpenMP and MPI. The N512 model scales effectively up to 12'244 cores and has now been successfully ported to PRACE TIER-0 systems (Curie and HERMIT), where it is operated in ensemble mode. Current developments include extensions to 17km and 12km grid spacing (N768 and N1024), which make use of up to 96 nodes on the new Met Office IBM-P7 system. The use of the next UM dynamical core, "EndGame", offers scaling improvements, with good performance on twice the current amount of cores, by altering the horizontal and vertical grid stagger, as well as eliminating the need for polar filtering.