Monday, November 20th 2017
"Summit" Supercomputer to Propel US Back to Number 1 in Top 500 by 2018
China has been increasingly - and steadily - gaining relevance in the supercomputing world, with most of the top-500 entries being controlled by that country. In fact, China can boast of having the number one supercomputer in the world, the Sunway TaihuLight, which can deliver 93 PetaFLOPS of computing power - just 3x more computational power than the second most powerful machine, China's own Tianhe-2). However, supercomputing, and the amount of money that's earned by selling processing slices of these supercomputers for private or state contractors, i a very attractive pull - especially considering the increasingly more expensive computational needs of the modern world.
The Summit is to be the United State's call to fame in that regard, bringing the country back to number one in raw, top-of-the-line single-machine supercomputing power. Summit is promising to more than double the PetaFLOPS of China's TaihuLight, to over 200 PetaFLOPs. That amounts to around 11x more processing grunt than its predecessor, the Titan, in a much smaller footprint - the Titan's 18,688 processing nodes will be condensed to just ~4,600 nodes on the Summit, with each node achieving around 40 TeraFLOPS of computing power. The hardware? IBM and NVIDIA, married in water-cooled nodes with the powerful GV100 accelerator that's still eluding us enthusiasts - but that's a question for another day.The silicon pieces that will make Summit tick and propel it to the, well, summit of the world's TOP 500 supercomputing lists is based on IBM's water-cooled Power Systems AC922 nodes - each of these packs 2x IBM POWER9 processors (which support up to eight memory channels, for a total of 16 channels per server that provide 340GB/s of aggregate bandwidth) alongside 6x Nvidia Volta GV100 GPUs. The Power9 processors are true beasts - they pack 8 billion transistors, and have up to 24 cores that can execute up to 96 threads. no, that's not a typo - it's thanks to the chips' SMT4 capabilities, which allow - you guessed it - up to four-way multithreading.Memory-wise, these processing nodes feature an aggregate of 512 GB of coherent DDR4 and HBM2 (High Bandwidth Memory) - along with a mind-blowing 1,600 GB of non-volatile RAM (out of a maximum 2,000 GB). These 1,600 GB of RAM per node serve as a burst buffer - they absorb bursts of data before transferring it to the remote primary storage pool, giving the system time to flush all of that data to more permanent, denser, slower memory banks.
However fast a supercomputer might be, one of the most common bottlenecks in today's supercomputing tasks is having enough inter-chip connectivity that can feed all processing hardware with information, while offering ways to store finished workloads to memory. For that, Summit will employ 96 lanes of PCIe 4.0 alongside a dual-port Mellanox EDR InfiniBand adapter, on which AMD has measured throughput of 392 Gb/s: just 8 Gb/s short of its theoretical maximum of 400 Gb/s. Naturally, NVIDIA's own NVLink 2.0 makes an appearance as well, alongside traditional PCIe 3.0. The NVLink interface provides 100GB/s of throughput for CPU-to-GPU and GPU-to-GPU traffic, with the GPUs being arranged in a dual-mesh design.~The Summit supercomputer will consume 15 MW of power (the site where it'll be deployed will be able to deliver up to 20 MW), which is on-par with China's Sunway - but remember, it more than doubles the peak PetaFlops from 93 to 200. For reference, the Titan system that came before it made do with 9 MW, but here, we're looking at around 35% increased power consumption for a roughly 11x increase in computing power - it's a vastly more efficient system. The entire Summit supercomputer will consume a space roughly the size of two basketball courts and require 136 miles of cabling - but even considering those massive engineering undertakings, it's on-schedule for deployment in 2018, and barring a top-secret supercomputer being in development by another country, it should propel the U.S. back into the supercomputing lead - at least for a time.
The Summit is to be the United State's call to fame in that regard, bringing the country back to number one in raw, top-of-the-line single-machine supercomputing power. Summit is promising to more than double the PetaFLOPS of China's TaihuLight, to over 200 PetaFLOPs. That amounts to around 11x more processing grunt than its predecessor, the Titan, in a much smaller footprint - the Titan's 18,688 processing nodes will be condensed to just ~4,600 nodes on the Summit, with each node achieving around 40 TeraFLOPS of computing power. The hardware? IBM and NVIDIA, married in water-cooled nodes with the powerful GV100 accelerator that's still eluding us enthusiasts - but that's a question for another day.The silicon pieces that will make Summit tick and propel it to the, well, summit of the world's TOP 500 supercomputing lists is based on IBM's water-cooled Power Systems AC922 nodes - each of these packs 2x IBM POWER9 processors (which support up to eight memory channels, for a total of 16 channels per server that provide 340GB/s of aggregate bandwidth) alongside 6x Nvidia Volta GV100 GPUs. The Power9 processors are true beasts - they pack 8 billion transistors, and have up to 24 cores that can execute up to 96 threads. no, that's not a typo - it's thanks to the chips' SMT4 capabilities, which allow - you guessed it - up to four-way multithreading.Memory-wise, these processing nodes feature an aggregate of 512 GB of coherent DDR4 and HBM2 (High Bandwidth Memory) - along with a mind-blowing 1,600 GB of non-volatile RAM (out of a maximum 2,000 GB). These 1,600 GB of RAM per node serve as a burst buffer - they absorb bursts of data before transferring it to the remote primary storage pool, giving the system time to flush all of that data to more permanent, denser, slower memory banks.
However fast a supercomputer might be, one of the most common bottlenecks in today's supercomputing tasks is having enough inter-chip connectivity that can feed all processing hardware with information, while offering ways to store finished workloads to memory. For that, Summit will employ 96 lanes of PCIe 4.0 alongside a dual-port Mellanox EDR InfiniBand adapter, on which AMD has measured throughput of 392 Gb/s: just 8 Gb/s short of its theoretical maximum of 400 Gb/s. Naturally, NVIDIA's own NVLink 2.0 makes an appearance as well, alongside traditional PCIe 3.0. The NVLink interface provides 100GB/s of throughput for CPU-to-GPU and GPU-to-GPU traffic, with the GPUs being arranged in a dual-mesh design.~The Summit supercomputer will consume 15 MW of power (the site where it'll be deployed will be able to deliver up to 20 MW), which is on-par with China's Sunway - but remember, it more than doubles the peak PetaFlops from 93 to 200. For reference, the Titan system that came before it made do with 9 MW, but here, we're looking at around 35% increased power consumption for a roughly 11x increase in computing power - it's a vastly more efficient system. The entire Summit supercomputer will consume a space roughly the size of two basketball courts and require 136 miles of cabling - but even considering those massive engineering undertakings, it's on-schedule for deployment in 2018, and barring a top-secret supercomputer being in development by another country, it should propel the U.S. back into the supercomputing lead - at least for a time.