Tuesday, February 18th 2020
UK Prepares $1.6 Billion for the Most Powerful Weather Forecasting Supercomputer
The UK government has set aside a budget of 1.2 billion GBP, which is roughly around 1.56 billion US Dollars. With this budget, the UK government plans to install the world's most powerful supercomputer used for weather forecasting in the year 2022. Previously, the UK government used three Cray XC40 supercomputers that are capable of achieving a maximum of 14 PetaFLOPs at its peak performance. The future system plans to take that number and make it look tiny. With plans to make it 20 times more powerful than the current machine, we can estimate that the future supercomputer will have above 200 PetaFLOPs of computing performance.
The supercomputer deployment will follow a series of cycles, where one is happening in 2022 and that supercomputer will be six times more powerful than the current solution. To get to that 20 times improvement, the supercomputer will get an upgrade over the next five years' time. While we do not know what will power the new machine, it will almost definitely be a CPU plus multi-GPU node configuration, as GPUs have gained a lot of traction in weather prediction models lately.
The supercomputer deployment will follow a series of cycles, where one is happening in 2022 and that supercomputer will be six times more powerful than the current solution. To get to that 20 times improvement, the supercomputer will get an upgrade over the next five years' time. While we do not know what will power the new machine, it will almost definitely be a CPU plus multi-GPU node configuration, as GPUs have gained a lot of traction in weather prediction models lately.

43 Comments on UK Prepares $1.6 Billion for the Most Powerful Weather Forecasting Supercomputer
I thought that AMD had that instruction set in the newer models of CPUs now? I thought that was the case anyways but I think there might be other versions of that set I guess?You're very lucky if you can get 1Gb for £65, I'm paying about half that for 30Mb!! :cry: The line we have at work is 1Gb up/down, I think that was something like £27k for so many years I think (dual lines different exchanges etc etc..
It needs to be EPYC + AMD GPUs.But its purpose will be climate simulation, not weather forecast.
The alternative to this approach for ML (which is one few use cases for AVX-512) is pairing the server CPUs with GPGPUs, preferably one with dedicated tensor cores (mostly NVIDIA at the moment) for higher efficiency.
Their current cluster uses E5-2695 v4, which is 18C. That generation offered SKUs with higher core count and higher frequencies (also both).
HPC systems are carefully selected and fine-tuned.
You can't assume everyone gets the 64-core EPYC right now so there's no reason to go with Intel.
Check the latest Top500 (54th). Out of 147 systems started in 2019, 99 don't use any accelerator - i.e. they're focused on CPU performance.
95 of them use Intel CPUs (from few generations).
Histogram of "Cores per Socket":
10: 1
12: 6
14: 6
16: 15
18: 5
20: 53
24: 5
28: 2
48: 2
Also, people somehow focused on AMD vs Intel, wheareas they may also go for something GPU-based or even ARM. Both approaches are being tested in weather modeling with good results.
The fact that your needs can be covered with 50% less sockets (when going 64-core EPYC) means indisputable savings if the 64-core EPYC is a product for you.
Down the ladder it's not that obvious, because you end up with the same number of sockets (nodes, blades) either way, so possible savings are limited.
And it's really not obvious that a EPYC one is cheaper just because EPYC CPU is cheaper. Other aspects may mean that a company like Cray is able to offer Intel variant for less - even if the CPU alone is more expensive.You're a shareholder or what? :o
finance.yahoo.com/quote/Amd?ltr=1
It's certainly not based on this Met Office contract - even if we assume AMD is sure to get it.
~30Mb/s is all you can hope for with fibre to the cabinet in the UK and BT/Openreach's decision to roll out FTTC as an upgrade to ADSL has probably set back the UK by a decade in the internet bandwidth race, IMO. I'm stuck on FTTC myself in london and at least 6 months away from a 1Gb/s HyperOptic FTTP installation.
We are, however, a minority. BT have installed FTTP to 4.1 million UK homes as of October 2019 and In addition to that, here are hybrid fibre/coax FTN networks. Take Virgin Media - who are already offering at least 108Mb/s to 75% of the UK, 53% of which is 360Mb/s capable network. Pricing is similar to Openreach FTTC, so although not quite FTTP it's good enough for most people at the moment. If you're not served by Virgin or Openreach's FTTP then you are probably one of the 15% who are either living out in the wilderness (and completely screwed until 5G can cover your internet bandwidth needs), or you're likely to be in the queue for an FTTP provider to cover your area.
As for business connections to the Met Office in Exeter, they are located are within half a kilometre of at least one Fibre ISP, and there are datacenters in the same radius for numerous bandwidth-hungry businesses including Amazon. I assume the Met office location was picked because it's such a well-connected, datacenter-heavy region, and the physical proximity will keep their internet costs down, not that it's remotely an issue given the running costs of the hardware itself!
In case you're wondering how I know this, my former employer (15 years ago) ran a datacenter in that area (Sowton Industrial Estate) and I spent years driving past all this every day.
I'm interested in finding out how they'd even put together a system like that, I just hope that they are going to use AMD hardware that's all :) It'll be a lovely push forward for them if they do I think :)It's even worse for me, aluminum cabling from the DSlam to my home.. I'm at about 700m I believe, I used to be getting upwards of 44Mb, now lucky if I can get 30Mb.. My sister who's up the road manages 65Mb and she and her husband are not even close to using it anywhere near to what I try and do with mine here :laugh: :cry: :laugh: :cry: If I don't laugh I'll be crying!! :( :(
Exeter isn't too far from me, but I'm not so fond about living in a busy city... But still, I digress :)
If only we had some more information about the build of the system etc.... I'd love to see what even goes in a system like this, let alone how it all works together :) Ah, being a hardware w^&%$ has it's fun spots I guess :)
I would like some good quality info like this:
- O.S. of the machine;
- The Software which is going to use;
- Who is going to program it, and how many parallel tasks can it run ?
Met Office is running an atmospheric model (currently one called Unified Model) and delivering weather forecasts. That's the primary purpose of their current infrastructure (Cray XC40) which this is going to replace.
OS - whatever the supplier provides for the cluster (Linux-based).
It will be used by Met staff, obviously...
I don't understand the "how many parallel tasks" part. 64-bit Linux is limited to 4194304 (2^22).
You can definitely forsee how nature behaves, it's called science, just as as physics can predict how galaxies behave, meteorology can predict how weather behaves. We don't live in fairy tale land, we can actually use our brains and tools to solve even the most complex problems. It's just another tool.
But, it's UK, so it's never going to happen.
if we want to buy 200 epyc cpu's we can get them in a week.
If we order 200 intel chips we wait 3 months minimum