AMD Instinct MI200: Dual-GPU Chiplet; CDNA2 Architecture; 128 GB HBM2E

[Raevenlord]

AMD today announced the debut of its 6 nm CDNA2 (Compute-DNA) architecture in the form of the MI200 family. The new, dual-GPU chiplet accelerator aims to lead AMD into a new era of High Performance Computing (HPC) applications, the high margin territory it needs to compete in for continued, sustainable growth. To that end, AMD has further improved on a matured, compute-oriented architecture born with Graphics Core Next (GCN) - and managed to improve performance while reducing total die size compared to its MI100 family.





AMD's MI250X accelerator features two compute dies with 58 Billion transistors built out of TSMC's 6 nm process. Each of these chips features a total of 110 Compute Units (CUs) for a total of 220 CUs on a single accelerator. The new CDNA2 architecture also incorporates new, improved Matrix Cores to the tune of 880 units (440 per chip). And as the MI250X is configured, that incredible amount of GPU power is paired with 128 GB of HBM2E memory, running at 3.2 TB/s. AMD's performance estimates against NVIDIA's current-gen A100 are blowouts. Compared to the A100, the MI250X is quoted as being: 4.9 times faster at FP64 vector compute; 2.5 times as fast in FP 32 Vector; 4.9 times faster in FP64 Matrix; a more meager 1.2 times faster performance on FP16 and BF16 Matrix operations; 1.6 times bigger memory capacity (128 GB on the MI 250X compared to the A100's 80 GB); and a 1.6 times faster memory bandwidth (though that's derived from the faster HBM2E memory).



The two CDNA2 dies are linked together AMD's Infinity Fabric, which makes its debut on a graphics architecture. This link provides a series of 25 Gbps links offering up to 100 GB/s of bi-directional bandwidth between both GPUs. There are eight available links in the MI200's distribution module - built to the specifications of an OAM (OCP Accelerator Module, where OCP stands for "Open Compute Platform") configuration. In total 800 GB/s of bandwidth are available for on-the fly communication between the two chiplets. AMD already announced that a PCIe version of the MI200 is launching in the future, catering to those who just want drop-in replacements or upgrades.



AMD's usage of TSMC's N6 fabrication technology can certainly account for part of the performance and die size improvement. As any manufacturer will do, AMD is employing yield optimizing efforts. This becomes clear when we look at the other product AMD is introducing alongside the MI250X: the MI250 accelerator. The MI250 takes a hit on computational resources by dropping from the MI250X's 110 CUs to 104 CUs per chiplet. That's actually the only change; it should be around 5% slower than the fully-enabled MI250X.



All in all, the MI200 series is a marked improvement for AMD performance-wise. And yet, NVIDIA is sure to announce their own next-generation compute solution soon. How will the updated, CDNA2-powered MI200 series stand?

View at TechPowerUp Main Site

 

[z1n0x]

It seems AMD is focusing on HPC. Double precision performance is impressive. On the low precision front (AI), it's barely faster than A100.

 

[Lycanwolfen]

Basicly Crossfire on a single card. Knew it would come to this. I'm sure the new Nvidia's Single Card will use SLI on a single card. Or Dual Core then Quad and so on.

 

[Fouquin]

Basicly Crossfire on a single card. Knew it would come to this. I'm sure the new Nvidia's Single Card will use SLI on a single card. Or Dual Core then Quad and so on.

Mmm, not quite. "Crossfire on a card" is something like what the old dual-GPUs used. A PLX bridging a master (GPU+compositing+bus control) and slave (GPU+bus) core onto a single card. This implementation isn't being used, instead it's now two GPUs and their associative memory sitting together on an interposer linked with a single fabric. It's not nearly as clunky as the old way of doing mGPU. These will appear as a single compute engine and not two split by a bridge. Much easier to address and control.

The two CDNA2 dies are linked together AMD's Infinity Fabric, which makes its debut on a graphics architecture.

Did this mean to say "... Infinity Fabric, which made its debut on a graphics architecture."? Because Infinity Fabric debuted on Vega in 2017, not here with CDNA2. It's a fundamental feature of GCN5+ interconnect logic. It would be surprising if it weren't featured on CDNA.

 

[mechtech]

And probably cheaper and more available than a 6800xt. .

 

[WhoDecidedThat]

128 GB of HBM2E memory, running at 3.2 GB/s

It's not 3.2 GB/s. Even my 150$ smartphone manages atleast 5x more than that in it's SoC. It's 3.2 TeraByte/sec aka 3200 GB/sec.

 

[wolf]

that incredible amount of GPU power is paired with 128 GB of HBM2E memory, running at 3.2 GB/s.

I believe that should read 3.2 TB/s?

edit:

blanarahul beat me to it!​

 

[Tom Yum]

It seems AMD is focusing on HPC. Double precision performance is impressive. On the low precision front (AI), it's barely faster than A100.

Makes sense given the design win for Frontier. Also makes sense for AMD to differentiate their designs from NVIDIA's so that they aren't targetting the exact same market (HPC vs AI). Both are big enough to be profitable.

 

[Mussels]

MI200 crysis benchmark time?

 

[R0H1T]

MI200 crysis benchmark time?

But can it run MS flight simulator 2020 at 16k 360fps?

 

[walker15130]

I believe that should read 3.2 TB/s?

Funny thing is that this memory is probably set to run at 3.2Gbps speeds (a.k.a. 3200MHz effective clock) with 8192bit overall bus width resulting in 3.27TB/s of bandwidth.

 

[Oberon]

These will appear as a single compute engine and not two split by a bridge. Much easier to address and control.

We aren't quite there yet. Each chip of the MCM is still exposed as a separate accelerator, but the topology of the system is such that there are multiple tiers of connectivity, with accelerators on the same MCM sharing the greatest bandwidth, followed by progressively less to accelerators located one or two hops away across the fabric on other OAM modules.

 

[medi01]

I don't get how that math works.
A100 - 54 billion transistors
MI200 - 58 billion transistors (29 + 29), yet it runs circles around A100.

It seems AMD is focusing on HPC. Double precision performance is impressive. On the low precision front (AI), it's barely faster than A100.

You mean 22% faster is "barely faster"?

 

[Fouquin]

We aren't quite there yet. Each chip of the MCM is still exposed as a separate accelerator, but the topology of the system is such that there are multiple tiers of connectivity, with accelerators on the same MCM sharing the greatest bandwidth, followed by progressively less to accelerators located one or two hops away across the fabric on other OAM modules.

I'm surprised that you believe we aren't there yet. Imagination has been doing MCM designs that address as a single unit for years, AMD does it with a much clunkier setup (and with the assistance of a bunch of MUXs) for Apple's accelerators, Intel is doing MCM for Sapphire Rapids that fits four tiles as a single accelerator. It's very much within the tech of today to handle multiple discrete dies as a single one.

 

[Oberon]

I'm surprised that you believe we aren't there yet. Imagination has been doing MCM designs that address as a single unit for years, AMD does it with a much clunkier setup (and with the assistance of a bunch of MUXs) for Apple's accelerators, Intel is doing MCM for Sapphire Rapids that fits four tiles as a single accelerator. It's very much within the tech of today to handle multiple discrete dies as a single one.

I'm not really commenting on whether it is possible, but rather that it is not how this particular product line works.

 

[Fouquin]

I'm not really commenting on whether it is possible, but rather that it is not how this particular product line works.

Interesting. I'll need to see if we get sampled any of these to find out for myself.

 

[Punkenjoy]

I don't get how that math works.
A100 - 54 billion transistors
MI200 - 58 billion transistors (29 + 29), yet it runs circles around A100.


You mean 22% faster is "barely faster"?

Right now, it "runs circles around A100" in theorical peak performance.

In reality? who know. The fact that Ampere have not that much less transistors for similar performance can mean that they choose to add a lot of in core memory. It's quite possible that MI200 is a hard to feed beast. Yes the high bandwidth can help, but still come with a latency disadvantage and will lead to a lot of unused cycles.

Vega problem was always about how hard it was to get maximum performance out of real workload.

 

[dragontamer5788]

It's not 3.2 GB/s. Even my 150$ smartphone manages atleast 5x more than that in it's SoC. It's 3.2 TeraByte/sec aka 3200 GB/sec.

Its really 2x 1.6 TB/s.

Which in some applications will be seen as 1.6 TB/s, while other applications may see 3.2 TB/s. It depends on how you organize your data, and whether or not you can split your data into two memory-systems that are independent of each other.

Given that infinity fabric is like 400GB/s between the dies, your average-case (assuming 25% of your data in on the "far" memory and 75% of your data is in the "near" memory) is 2.0 TB/s, and only if one of the processors is near-idle. If you can split your data to be 100% in near-memory and not worry about the far-memory at all and have perfect parallelism, you can theoretically get to 3.2 TB.

Not that 1.6 TB/s is slow. But... given the "crossfire-like" setup here, I think its misleading to call this 3.2 TB of bandwidth.

 

[TheoneandonlyMrK]

Doooooo want.

 

[dragontamer5788]

Doooooo want.

MI100 is quoted at $8000 though. I'd expect the PCIe MI200 to be at least $8000, maybe $10k and beyond.

Hmmm... I guess I'd "want" the OAM system, but who the heck would even sell you an OAM motherboard / system to play with? Those have to be incredibly expensive (thinking like $2000+ motherboards alone).

NVidia's DGX systems are $200,000+ computers. That's what I'd expect OAM-systems to cost at a minimum, maybe $200k to $1MM. Cache-coherent fabrics between CPU and GPU is just going to be expensive as all heck.

------------

Even the PCIe card is probably going to be outside the cost of any reasonable hobbyist.

 

[TheoneandonlyMrK]

MI100 is quoted at $8000 though. I'd expect the PCIe MI200 to be at least $8000, maybe $10k and beyond.

Hmmm... I guess I'd "want" the OAM system, but who the heck would even sell you an OAM motherboard / system to play with? Those have to be incredibly expensive (thinking like $2000+ motherboards alone).

NVidia's DGX systems are $200,000+ computers. That's what I'd expect OAM-systems to cost at a minimum, maybe $200k to $1MM. Cache-coherent fabrics between CPU and GPU is just going to be expensive as all heck.

------------

Even the PCIe card is probably going to be outside the cost of any reasonable hobbyist.

I accept this reality, just like in the same reality I also dooo want a Rs7 GT etron.

And btw assuming anything is beyond someone you don't know is a bit lame, I do get to mess with some nice stuff at work, and home, it's just not always my own.

And is coincidentally why I build PC for others on the side , joy ,and obviously those 3dmark achievements won't get themselves.

 

[dragontamer5788]

And btw assuming anything is beyond someone you don't know is a bit lame, I do get to mess with some nice stuff at work, and home, it's just not always my own.

I've definitely had bosses who were open to the idea of buying $200,000+ computers for learning purposes for the team. But... that's kind of different than putting one of these things in your garage, ya know? I'd also assume that professors at universities also get access to these systems (as they'd nominally be for students to learn off of). But its not like, an individual hobbyist purchase.

Most likely, a "small person" would have access to these things if a cloud-provider started renting out some time to them. Either that, or university classes methinks. That's just how to make the $200,000 to $20,000,000 equipment work out for a community.

Its not so much "out of reach" as much as a community purchase. Either the community at your work, or some other learning-oriented community (technical school, university, colleges).

-----

I guess there are some very rich people out there who might be able to afford one of these things in their garage though. But if someone is that rich, I would hope that they join a university and allow the purchase to be used by students as well Acts of charity and all that, lol.

 

[TheoneandonlyMrK]

I've definitely had bosses who were open to the idea of buying $200,000+ computers for learning purposes for the team. But... that's kind of different than putting one of these things in your garage, ya know? I'd also assume that professors at universities also get access to these systems (as they'd nominally be for students to learn off of). But its not like, an individual hobbyist purchase.

Most likely, a "small person" would have access to these things if a cloud-provider started renting out some time to them. Either that, or university classes methinks. That's just how to make the $200,000 to $20,000,000 equipment work out for a community.

Its not so much "out of reach" as much as a community purchase. Either the community at your work, or some other learning-oriented community (technical school, university, colleges).

-----

I guess there are some very rich people out there who might be able to afford one of these things in their garage though. But if someone is that rich, I would hope that they join a university and allow the purchase to be used by students as well Acts of charity and all that, lol.

All top tech gets old eventually I have Tesla's and other exotics of the past, plus I did say want not I'm getting because I am not but damnnnnmn dooo want!!?.

I'm not sure what your problem is actually as I said I want all sorts of shit I am ok with never owning, don't tell , your different aren't you!.?

 

[medi01]

Right now, it "runs circles around A100" in theorical peak performance.

In reality? who know. The fact that Ampere have not that much less transistors for similar performance can mean that they choose to add a lot of in core memory. It's quite possible that MI200 is a hard to feed beast. Yes the high bandwidth can help, but still come with a latency disadvantage and will lead to a lot of unused cycles.

Vega problem was always about how hard it was to get maximum performance out of real workload.

Figures that it is compared to is also theoretical peak performance, is it not?
I see your point though, recalling 3090's figures.

MI100 is quoted at $8000 though. I'd expect the PCIe MI200 to be at least $8000, maybe $10k and beyond.

A100 were sold in packs of 10, if I'm not mistaken, for $200k. I don't see why AMD would ask half of that sum for a vastly faster product.

 

[Lycanwolfen]

Mmm, not quite. "Crossfire on a card" is something like what the old dual-GPUs used. A PLX bridging a master (GPU+compositing+bus control) and slave (GPU+bus) core onto a single card. This implementation isn't being used, instead it's now two GPUs and their associative memory sitting together on an interposer linked with a single fabric. It's not nearly as clunky as the old way of doing mGPU. These will appear as a single compute engine and not two split by a bridge. Much easier to address and control.



Did this mean to say "... Infinity Fabric, which made its debut on a graphics architecture."? Because Infinity Fabric debuted on Vega in 2017, not here with CDNA2. It's a fundamental feature of GCN5+ interconnect logic. It would be surprising if it weren't featured on CDNA.

View attachment 224420

Still Dual GPU's acting as one. Yes faster with the new fabric but still basic Idea. When you get into 4k or even 8k gaming single card cannot handle it pushing over 100 fps. I tried that with one 1070ti forget it. My two in SLI can push 100 fps easy. But since Nvidia and AMD dropped that tech. I bought a 3080TI and tried it at 4k could not push 100 fps constantly. Ya it looks great but my eyes can see the lag and frame buffering trying to keep up. Now lucky friend of mine has two 3090's in SLI and man 8k res at 150 FPS looks soo clean and perfect. But I do not have 5 grand lying around to afford such nice things.