• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

"Summit" Supercomputer to Propel US Back to Number 1 in Top 500 by 2018

Raevenlord

News Editor
Joined
Aug 12, 2016
Messages
3,755 (1.35/day)
Location
Portugal
System Name The Ryzening
Processor AMD Ryzen 9 5900X
Motherboard MSI X570 MAG TOMAHAWK
Cooling Lian Li Galahad 360mm AIO
Memory 32 GB G.Skill Trident Z F4-3733 (4x 8 GB)
Video Card(s) Gigabyte RTX 3070 Ti
Storage Boot: Transcend MTE220S 2TB, Kintson A2000 1TB, Seagate Firewolf Pro 14 TB
Display(s) Acer Nitro VG270UP (1440p 144 Hz IPS)
Case Lian Li O11DX Dynamic White
Audio Device(s) iFi Audio Zen DAC
Power Supply Seasonic Focus+ 750 W
Mouse Cooler Master Masterkeys Lite L
Keyboard Cooler Master Masterkeys Lite L
Software Windows 10 x64
China has been increasingly - and steadily - gaining relevance in the supercomputing world, with most of the top-500 entries being controlled by that country. In fact, China can boast of having the number one supercomputer in the world, the Sunway TaihuLight, which can deliver 93 PetaFLOPS of computing power - just 3x more computational power than the second most powerful machine, China's own Tianhe-2). However, supercomputing, and the amount of money that's earned by selling processing slices of these supercomputers for private or state contractors, i a very attractive pull - especially considering the increasingly more expensive computational needs of the modern world.

The Summit is to be the United State's call to fame in that regard, bringing the country back to number one in raw, top-of-the-line single-machine supercomputing power. Summit is promising to more than double the PetaFLOPS of China's TaihuLight, to over 200 PetaFLOPs. That amounts to around 11x more processing grunt than its predecessor, the Titan, in a much smaller footprint - the Titan's 18,688 processing nodes will be condensed to just ~4,600 nodes on the Summit, with each node achieving around 40 TeraFLOPS of computing power. The hardware? IBM and NVIDIA, married in water-cooled nodes with the powerful GV100 accelerator that's still eluding us enthusiasts - but that's a question for another day.





The silicon pieces that will make Summit tick and propel it to the, well, summit of the world's TOP 500 supercomputing lists is based on IBM's water-cooled Power Systems AC922 nodes - each of these packs 2x IBM POWER9 processors (which support up to eight memory channels, for a total of 16 channels per server that provide 340GB/s of aggregate bandwidth) alongside 6x Nvidia Volta GV100 GPUs. The Power9 processors are true beasts - they pack 8 billion transistors, and have up to 24 cores that can execute up to 96 threads. no, that's not a typo - it's thanks to the chips' SMT4 capabilities, which allow - you guessed it - up to four-way multithreading.



Memory-wise, these processing nodes feature an aggregate of 512 GB of coherent DDR4 and HBM2 (High Bandwidth Memory) - along with a mind-blowing 1,600 GB of non-volatile RAM (out of a maximum 2,000 GB). These 1,600 GB of RAM per node serve as a burst buffer - they absorb bursts of data before transferring it to the remote primary storage pool, giving the system time to flush all of that data to more permanent, denser, slower memory banks.

However fast a supercomputer might be, one of the most common bottlenecks in today's supercomputing tasks is having enough inter-chip connectivity that can feed all processing hardware with information, while offering ways to store finished workloads to memory. For that, Summit will employ 96 lanes of PCIe 4.0 alongside a dual-port Mellanox EDR InfiniBand adapter, on which AMD has measured throughput of 392 Gb/s: just 8 Gb/s short of its theoretical maximum of 400 Gb/s. Naturally, NVIDIA's own NVLink 2.0 makes an appearance as well, alongside traditional PCIe 3.0. The NVLink interface provides 100GB/s of throughput for CPU-to-GPU and GPU-to-GPU traffic, with the GPUs being arranged in a dual-mesh design.~



The Summit supercomputer will consume 15 MW of power (the site where it'll be deployed will be able to deliver up to 20 MW), which is on-par with China's Sunway - but remember, it more than doubles the peak PetaFlops from 93 to 200. For reference, the Titan system that came before it made do with 9 MW, but here, we're looking at around 35% increased power consumption for a roughly 11x increase in computing power - it's a vastly more efficient system. The entire Summit supercomputer will consume a space roughly the size of two basketball courts and require 136 miles of cabling - but even considering those massive engineering undertakings, it's on-schedule for deployment in 2018, and barring a top-secret supercomputer being in development by another country, it should propel the U.S. back into the supercomputing lead - at least for a time.

View at TechPowerUp Main Site
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,259 (4.65/day)
Location
IA, USA
System Name BY-2021
Processor AMD Ryzen 7 5800X (65w eco profile)
Motherboard MSI B550 Gaming Plus
Cooling Scythe Mugen (rev 5)
Memory 2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s) AMD Radeon RX 7900 XT
Storage Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse Nixeus REVEL-X
Keyboard Tesoro Excalibur
Software Windows 10 Home 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
Nope, but it can probably tell you how poorly you run Crysis. Because maths.


The water cooled aspect of this machine is the most interesting and likely where a lot of the power savings comes from. I wonder if they're just using giant external radiators or if there's some kind of phase change system involved somewhere.

Looks like only two 40mm fans per rack in the power supply. I wonder why they don't make the power system 12v DC vastly simplifying the power system in each unit.
 
Joined
Jan 29, 2012
Messages
6,402 (1.44/day)
Location
Florida
System Name natr0n-PC
Processor Ryzen 5950x/5600x
Motherboard B450 AORUS M
Cooling EK AIO - 6 fan action
Memory Patriot - Viper Steel DDR4 (B-Die)(4x8GB)
Video Card(s) EVGA 3070ti FTW
Storage Various
Display(s) PIXIO IPS 240Hz 1080P
Case Thermaltake Level 20 VT
Audio Device(s) LOXJIE D10 + Kinter Amp + 6 Bookshelf Speakers Sony+JVC+Sony
Power Supply Super Flower Leadex III ARGB 80+ Gold 650W
Software XP/7/8.1/10
Benchmark Scores http://valid.x86.fr/79kuh6
Do they use pool pumps ?
 

bug

Joined
May 22, 2015
Messages
13,163 (4.07/day)
Processor Intel i5-12600k
Motherboard Asus H670 TUF
Cooling Arctic Freezer 34
Memory 2x16GB DDR4 3600 G.Skill Ripjaws V
Video Card(s) EVGA GTX 1060 SC
Storage 500GB Samsung 970 EVO, 500GB Samsung 850 EVO, 1TB Crucial MX300 and 2TB Crucial MX500
Display(s) Dell U3219Q + HP ZR24w
Case Raijintek Thetis
Audio Device(s) Audioquest Dragonfly Red :D
Power Supply Seasonic 620W M12
Mouse Logitech G502 Proteus Core
Keyboard G.Skill KM780R
Software Arch Linux + Win10
It's the software that runs on these things that make you #1 or not, not the pure computing power. But you read so little about the software...
 
Joined
Aug 20, 2007
Messages
20,710 (3.41/day)
System Name Pioneer
Processor Ryzen R9 7950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage 2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches
Software Windows 11 Enterprise (legit), Gentoo Linux x64
It's the software that runs on these things that make you #1 or not, not the pure computing power. But you read so little about the software...

Uh, no. The TOP500 has no say at all about software and literally no shits are given on that front. It's all about the hardware and making that hardware talk for PFLOPS, baby.
 
Joined
Jul 5, 2013
Messages
25,559 (6.52/day)
Uh, no. The TOP500 has no say at all about software and literally no shits are given on that front. It's all about the hardware and making that hardware talk for PFLOPS, baby.
Hardware does nothing without software to instruct it..
 
Joined
Aug 20, 2007
Messages
20,710 (3.41/day)
System Name Pioneer
Processor Ryzen R9 7950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage 2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches
Software Windows 11 Enterprise (legit), Gentoo Linux x64
Hardware does nothing without software to instruct it..

I'm very aware. You are talking to a programmer.

My point is that the software is neigh near irrelevant compared to the hardware though. He made it sound like it's literally the software the makes or breaks the machines standing position.
 

cdawall

where the hell are my stars
Joined
Jul 23, 2006
Messages
27,680 (4.29/day)
Location
Houston
System Name All the cores
Processor 2990WX
Motherboard Asrock X399M
Cooling CPU-XSPC RayStorm Neo, 2x240mm+360mm, D5PWM+140mL, GPU-2x360mm, 2xbyski, D4+D5+100mL
Memory 4x16GB G.Skill 3600
Video Card(s) (2) EVGA SC BLACK 1080Ti's
Storage 2x Samsung SM951 512GB, Samsung PM961 512GB
Display(s) Dell UP2414Q 3840X2160@60hz
Case Caselabs Mercury S5+pedestal
Audio Device(s) Fischer HA-02->Fischer FA-002W High edition/FA-003/Jubilate/FA-011 depending on my mood
Power Supply Seasonic Prime 1200w
Mouse Thermaltake Theron, Steam controller
Keyboard Keychron K8
Software W10P
I just want 3 gv100 cards for my machine.
 
Joined
Dec 10, 2015
Messages
545 (0.18/day)
Location
Here
System Name Skypas
Processor Intel Core i7-6700
Motherboard Asus H170 Pro Gaming
Cooling Cooler Master Hyper 212X Turbo
Memory Corsair Vengeance LPX 16GB
Video Card(s) MSI GTX 1060 Gaming X 6GB
Storage Corsair Neutron GTX 120GB + WD Blue 1TB
Display(s) LG 22EA63V
Case Corsair Carbide 400Q
Power Supply Seasonic SS-460FL2 w/ Deepcool XFan 120
Mouse Logitech B100
Keyboard Corsair Vengeance K70
Software Windows 10 Pro (to be replaced by 2025)
That is an expensive setup for mining coin
/joke

With Nvidia prefer it's product to be paired with IBM Power, it's another reason for Intel to develop it's own discrete GPU
 
Joined
Jul 5, 2013
Messages
25,559 (6.52/day)
I'm very aware. You are talking to a programmer. My point is that the software is neigh near irrelevant compared to the hardware though. He made it sound like it's literally the software the makes or breaks the machines standing position.
Ah, point taken.
That is an expensive setup for mining coin /joke
Joking aside, that supercomputer could very likely crank out bitcoin in minutes instead of days/weeks. But good grief the power bill!
With Nvidia prefer it's product to be paired with IBM Power, it's another reason for Intel to develop it's own discrete GPU
That's an assumption which isn't supported by real-world fact. NVidia has it's hands in a lot of technological sectors. Their are not going to abandon the gaming and PC markets. They are simply expanding into other markets to which they have something good to offer. Their GPGPU offerings are very powerful and can be tailored fit a number of computing requirements. AMD, likewise has much to offer and are gearing up to do so.
 
Joined
Apr 12, 2013
Messages
6,728 (1.68/day)
I just want 3 gv100 cards for my machine.
But can it run crysis @4k :rolleyes:

Seriously though, weren't there rumors that GV100 (compute oriented) can't run games?
That is an expensive setup for mining coin
/joke

With Nvidia prefer it's product to be paired with IBM Power, it's another reason for Intel to develop it's own discrete GPU
I wonder if anyone ever thought of that, you could probably mine billions of $ with this kind of setup during the early days of BTC or ETH.
 
Last edited:
Joined
Feb 18, 2012
Messages
2,715 (0.61/day)
System Name MSI GP76
Processor intel i7 11800h
Cooling 2 laptop fans
Memory 32gb of 3000mhz DDR4
Video Card(s) Nvidia 3070
Storage x2 PNY 8tb cs2130 m.2 SSD--16tb of space
Display(s) 17.3" IPS 1920x1080 240Hz
Power Supply 280w laptop power supply
Mouse Logitech m705
Keyboard laptop keyboard
Software lots of movies and Windows 10 with win 7 shell
Benchmark Scores Good enough for me
I would love to mine with this machine.
 
Joined
Sep 25, 2012
Messages
2,074 (0.49/day)
Location
Jacksonhole Florida
System Name DEVIL'S ABYSS
Processor i7-4790K@4.6 GHz
Motherboard Asus Z97-Deluxe
Cooling Corsair H110 (2 x 140mm)(3 x 140mm case fans)
Memory 16GB Adata XPG V2 2400MHz
Video Card(s) EVGA 780 Ti Classified
Storage Intel 750 Series 400GB (AIC), Plextor M6e 256GB (M.2), 13 TB storage
Display(s) Crossover 27QW (27"@ 2560x1440)
Case Corsair Obsidian 750D Airflow
Audio Device(s) Realtek ALC1150
Power Supply Cooler Master V1000
Mouse Ttsports Talon Blu
Keyboard Logitech G510
Software Windows 10 Pro x64 version 1803
Benchmark Scores Passmark CPU score = 13080
Joined
Aug 20, 2007
Messages
20,710 (3.41/day)
System Name Pioneer
Processor Ryzen R9 7950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage 2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches
Software Windows 11 Enterprise (legit), Gentoo Linux x64
I would love to mine with this machine.

Pay the powerbill and you might take that back, depending on how optimized it is for it. ;)
 
Joined
Apr 12, 2013
Messages
6,728 (1.68/day)
I'm betting that is going to be a thing. That's what many supercomputer owners do.
Not sure if you're being sarcastic or serious, if serious that'd be great. Even a handful of nodes could do certain tasks pretty quickly, assuming that it's feasible.
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,259 (4.65/day)
Location
IA, USA
System Name BY-2021
Processor AMD Ryzen 7 5800X (65w eco profile)
Motherboard MSI B550 Gaming Plus
Cooling Scythe Mugen (rev 5)
Memory 2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s) AMD Radeon RX 7900 XT
Storage Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse Nixeus REVEL-X
Keyboard Tesoro Excalibur
Software Windows 10 Home 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
Can you rent this, like AWS or Azure IIRC?
I don't think they request money. They need to know what project you're going to run and how much time it requires. Then they work with you to optimize your code for the hardware. Then they schedule you time on the machine to execute your code. National Laboratories supercomputers are for science.
 
Joined
Apr 12, 2013
Messages
6,728 (1.68/day)
I don't think they request money. They need to know what project you're going to run and how much time it requires. Then they work with you to optimize your code for the hardware. Then they schedule you time on the machine to execute your code. National Laboratories supercomputers are for science.
I understand that, I was just wondering if they could do that without making money, probably relying solely(?) on grants.
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,259 (4.65/day)
Location
IA, USA
System Name BY-2021
Processor AMD Ryzen 7 5800X (65w eco profile)
Motherboard MSI B550 Gaming Plus
Cooling Scythe Mugen (rev 5)
Memory 2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s) AMD Radeon RX 7900 XT
Storage Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse Nixeus REVEL-X
Keyboard Tesoro Excalibur
Software Windows 10 Home 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
Pretty sure the National Laboratories are run under Department of Energy's budget. When not working on external projects, they're running simulations on the US nuclear arsenal.
 
Joined
Jan 5, 2006
Messages
17,693 (2.66/day)
System Name AlderLake / Laptop
Processor Intel i7 12700K P-Cores @ 5Ghz / Intel i3 7100U
Motherboard Gigabyte Z690 Aorus Master / HP 83A3 (U3E1)
Cooling Noctua NH-U12A 2 fans + Thermal Grizzly Kryonaut Extreme + 5 case fans / Fan
Memory 32GB DDR5 Corsair Dominator Platinum RGB 6000MHz CL36 / 8GB DDR4 HyperX CL13
Video Card(s) MSI RTX 2070 Super Gaming X Trio / Intel HD620
Storage Samsung 980 Pro 1TB + 970 Evo 500GB + 850 Pro 512GB + 860 Evo 1TB x2 / Samsung 256GB M.2 SSD
Display(s) 23.8" Dell S2417DG 165Hz G-Sync 1440p / 14" 1080p IPS Glossy
Case Be quiet! Silent Base 600 - Window / HP Pavilion
Audio Device(s) Panasonic SA-PMX94 / Realtek onboard + B&O speaker system / Harman Kardon Go + Play / Logitech G533
Power Supply Seasonic Focus Plus Gold 750W / Powerbrick
Mouse Logitech MX Anywhere 2 Laser wireless / Logitech M330 wireless
Keyboard RAPOO E9270P Black 5GHz wireless / HP backlit
Software Windows 11 / Windows 10
Benchmark Scores Cinebench R23 (Single Core) 1936 @ stock Cinebench R23 (Multi Core) 23006 @ stock

bug

Joined
May 22, 2015
Messages
13,163 (4.07/day)
Processor Intel i5-12600k
Motherboard Asus H670 TUF
Cooling Arctic Freezer 34
Memory 2x16GB DDR4 3600 G.Skill Ripjaws V
Video Card(s) EVGA GTX 1060 SC
Storage 500GB Samsung 970 EVO, 500GB Samsung 850 EVO, 1TB Crucial MX300 and 2TB Crucial MX500
Display(s) Dell U3219Q + HP ZR24w
Case Raijintek Thetis
Audio Device(s) Audioquest Dragonfly Red :D
Power Supply Seasonic 620W M12
Mouse Logitech G502 Proteus Core
Keyboard G.Skill KM780R
Software Arch Linux + Win10
Uh, no. The TOP500 has no say at all about software and literally no shits are given on that front. It's all about the hardware and making that hardware talk for PFLOPS, baby.
That's what I said. It's easier to make news about "my PFLOPS is bigger than your PFLOPS". But the exciting part about supercomputers is elsewhere.
 
Top