• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Intel Announces New Mesh Interconnect For Xeon Scalable, Skylake-X Processors

Joined
Aug 20, 2007
Messages
20,767 (3.41/day)
System Name Pioneer
Processor Ryzen R9 7950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage 2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches + PBT DS keycaps
Software Gentoo Linux x64
Intel's "Xeon Scalable" lineup is designed to compete directly with AMD's Naples platform. Naples, a core-laden, high performance server platform that relies deeply on linking multiple core complexes together via AMD's own HyperTransport derived Infinity Fabric Interconnect has given intel some challenges in terms of how to structure its own high-core count family of devices. This has led to a new mesh-based interconnect technology from Intel.



When linking multiple core complexes together, Intel has traditionally relied on its QPI (Quick Path Interconnect) Interconnect, but it has its own limitations. For starters, QPI is a point to point technology, and it is inherently unsuitable for linking cores to cores in a mesh topology as is needed when random cores want to address each other. To work, you'd need to create a QPI link from every core to every other core on the chip, and that would be a waste of resources. Historically, to fill that gap Intel has used a "Ring bus." A ring bus functions similar to a token ring bus if you want to think of it in simple "old networking" terms. Basically, when data is transmitted, it must be passed around the bus like a token going from one speaker to the next. Each clock cycle, the bus can shift the data one way or another, but if you want to go say for example from one core to the farthest away one possible, latency suffers. In other words: This works fine for small dies, but as Intel seeks to create some true monsters, it's no longer enough.

There are some caveats to the above explanation (Intel's ring bus can have more than one "token" at a time in the analogy, and it's bidirectional, meaning data can travel both ways), but it still doesn't change the fact Intel is feeling this designs limits.

Enter the Mesh Interconnect:
Technically, this new Mesh Interconnect tech from Intel was introduced with the Xeon Phi Knight's Landing based products, but those are very exclusive, high end parts even for us mere enthusiast mortals. This launch represents the technology trickling down to a more mainstream market. To explain the tech, you simply have to forget everything you know about the ring bus chips used to use, and realize that now in the simplest chip implementations, each core has a direct "phone line" to every neighboring chip. It's a full mesh topology with all its benefits. In more complex arrangements, the "direct line to every chip" arrangement of course becomes uneconomical again, being it would require a harsh amount of resources for any chip maker to implement on massive chips the likes of which Naples and Xeon Scalable talk about. So instead, we go back to a simpler mesh topology, rows and columns.

Take this graphic, from Xeon Phi, to get an idea of how it works:



Basically, it's a grid with a full XY routing system. This means rather than run around a huge bus-like circle, they can go through a much more compact "cube" topology and save time. In a simple 3x3 cube example, whereas it would take a total of 8 clock cycles on an equivalent ringbus for communications to reach the farthest core, the most it can take on a Mesh Interconnect based core is 4. (-X-X+Y+Y is how the worst case routing would look, if you've been following this in that level of detail).

If a lot of this goes over your head, don't worry, it's pretty technical low level stuff. The bottom line is Intel has an Interconnect that can compete roughly with AMD's Infinity Fabric in its own respect, and given Intel's larger core count on each die, AMD may be wise to keep a watchful eye on that fact.

The Mesh interconnect is scheduled to debut in the much more "mainstream" (if you can call them that) markets of Xeon Scalable and Skylake-X, due to launch soon this year.

View at TechPowerUp Main Site
 
Last edited:
Joined
Sep 15, 2007
Messages
3,944 (0.65/day)
Location
Police/Nanny State of America
Processor OCed 5800X3D
Motherboard Asucks C6H
Cooling Air
Memory 32GB
Video Card(s) OCed 6800XT
Storage NVMees
Display(s) 32" Dull curved 1440
Case Freebie glass idk
Audio Device(s) Sennheiser
Power Supply Don't even remember
Intel is really going with the full on Chinese approach to business, eh? :D

Their decisions sure are interesting. They could have done this years ago, but didn't. Now, it's all, "See, we have this stuff, too, look at us."

It must only be a matter of time (new architecture) until they direct copy and start slapping together MCMs, since yields are killing them on these large core counts.
 
Last edited:

cdawall

where the hell are my stars
Joined
Jul 23, 2006
Messages
27,680 (4.27/day)
Location
Houston
System Name All the cores
Processor 2990WX
Motherboard Asrock X399M
Cooling CPU-XSPC RayStorm Neo, 2x240mm+360mm, D5PWM+140mL, GPU-2x360mm, 2xbyski, D4+D5+100mL
Memory 4x16GB G.Skill 3600
Video Card(s) (2) EVGA SC BLACK 1080Ti's
Storage 2x Samsung SM951 512GB, Samsung PM961 512GB
Display(s) Dell UP2414Q 3840X2160@60hz
Case Caselabs Mercury S5+pedestal
Audio Device(s) Fischer HA-02->Fischer FA-002W High edition/FA-003/Jubilate/FA-011 depending on my mood
Power Supply Seasonic Prime 1200w
Mouse Thermaltake Theron, Steam controller
Keyboard Keychron K8
Software W10P
Intel is really going with the full on Chinese approach to business, eh? :D

Their decisions sure are interesting. They could have done this years ago, but didn't. Now, it's all, "See, we have this stuff, too, look at us."

More like hey look at all this stuff we have been sitting on for 6 years while AMD played with an awful design.
 
Joined
Mar 18, 2008
Messages
5,717 (0.97/day)
System Name Virtual Reality / Bioinformatics
Processor Undead CPU
Motherboard Undead TUF X99
Cooling Noctua NH-D15
Memory GSkill 128GB DDR4-3000
Video Card(s) EVGA RTX 3090 FTW3 Ultra
Storage Samsung 960 Pro 1TB + 860 EVO 2TB + WD Black 5TB
Display(s) 32'' 4K Dell
Case Fractal Design R5
Audio Device(s) BOSE 2.0
Power Supply Seasonic 850watt
Mouse Logitech Master MX
Keyboard Corsair K70 Cherry MX Blue
VR HMD HTC Vive + Oculus Quest 2
Software Windows 10 P
Inside Intel:

Boss: Shit, AMD's new prcessors may screw up our financial report. Any ideas?

PR: more advertising? Pay editors and tech sites to spread FUD for AMD?

Management: Cut some jobs?

Engineers: Give us some resources so we can build better stuff?


Boss: Full on advertising mode PR team; and you can fire more engineers, management team. Let's take some bad Xeons and call them our I9
 
Joined
Sep 26, 2012
Messages
860 (0.20/day)
Location
Australia
System Name ATHENA
Processor AMD 7950X
Motherboard ASUS Crosshair X670E Extreme
Cooling Noctua NH-D15S, 7 x Noctua NF-A14 industrialPPC IP67 2000RPM
Memory 2x32GB Trident Z RGB 6000Mhz CL30
Video Card(s) ASUS 4090 Strix
Storage 3 x Kingston Fury 4TB, 4 x Samsung 870 QVO
Display(s) Alienware AW3821DW, Wacom Cintiq Pro 15
Case Fractal Design Torrent
Audio Device(s) Topping A90/D90 MQA, Fluid FPX7 Fader Pro, Beyerdynamic T1 G2, Beyerdynamic MMX300
Power Supply ASUS THOR 1600T
Mouse Xtrfy MZ1 - Zy' Rail, Logitech MX Vertical, Logitech MX Master 3
Keyboard Logitech G915 TKL
VR HMD Oculus Quest 2
Software Windows 11 + OpenSUSE MicroOS
Doesn't solve their yield problem, but will definitely help with their multi ring bus designs.

Wonder how much of the die is dedicated to cache coherency in this arrangement then? As thats Intel's biggest killer of clocks in high density parts.
 
Joined
Mar 7, 2011
Messages
3,925 (0.82/day)
One bad product release after another now onto ripping off competitors.

1519548_10200526739810488_1909832728_o.jpg
 
D

Deleted member 172152

Guest
Essentially they're making their cpu architecture better, but are now giving amd free bonus support and that means amd will now get even closer to intel! Also, unless intel has connected mesh speed to ram speed like amd, there is potentially less need for fast ram and low latency, possibly giving amd the edge in the memory department, which would give amd even more chance of catching up and cashing in!

WTF intel?!?!?!?!?!?!
 
Last edited by a moderator:
Joined
Jun 28, 2016
Messages
3,595 (1.26/day)
One bad product release after another now onto ripping off competitors.
Ripping off?
Even in this article you'll find information that Intel has already used this tech in 2016.

You're such an Intel fanboy. Get a grip.
 
Joined
Sep 11, 2015
Messages
624 (0.20/day)
Ripping off?
Even in this article you'll find information that Intel has already used this tech in 2016.

You're such an Intel fanboy. Get a grip.
Intel fanboy ripps on Intel... typo?

But they're obviously just parroting what AMD is doing at this point. You don't have to be a fanboy to see what's so obvious.
 
Joined
Mar 24, 2017
Messages
123 (0.05/day)
Location
Italy
This is not an industrial seecret, it's a simple mesh interconnection.
They are simply doing what they should have done years ago but didn't because the money was flowing in anyway.
Now they are investing money to make their product better and more competitive (with a very obvious and cheap way to rise performance).

Let's all rejoice AMD hit good and hard with a great product.
 
Joined
May 10, 2007
Messages
1,420 (0.23/day)
Location
London, England
Intel fanboy ripps on Intel... typo?

But they're obviously just parroting what AMD is doing at this point. You don't have to be a fanboy to see what's so obvious.

They've been using a mesh since the Xeon Phi Knights Landing, says so in this very article:
Technically, this new Mesh Interconnect tech from Intel was introduced with the Xeon Phi Knight's Landing based products

If you're going to try something new, there's no better (worse) place to start than an expensive, targeted product used by the most exacting customers, on a highly complex core. If it works for a 72-core processor, it's probably okay on a 22-core or less processor.
 
Joined
Jun 28, 2016
Messages
3,595 (1.26/day)
Intel fanboy ripps on Intel... typo?
No. @Chaitanya suggested that in few months Intel designed a solution that took AMD few years. Clearly, he must think of Intel as god of electronics.
But they're obviously just parroting what AMD is doing at this point. You don't have to be a fanboy to see what's so obvious.
In marketing? Yes.
Mesh interconnect was already used in a 2016 Xeon Phi, but was merely a complicated technical feature - hidden deeply in the architecture documentation. Now they've pushed it into spotlight - exactly what AMD did with Infinity Matrix.

Personally, I'm not a huge fan of what's happening. I don't like the whole Ryzen marketing campaign (flashy, gaming-themed) and I really hope Intel will keep their cool, business-oriented image. But I guess this CPU segment is meant for geeks mostly (like people on TPU :) ) and they need to push some technical stuff to the surface. Enough to generate topics like this one, but not very technical, so that you don't have to be a physicist or electrical engineer to participate in a discussion.

To begin with, I like the fact that Intel kept the original name (mesh interconnect) from Xeons and didn't replace it with something more "sci-fi". Honestly, "Infinity Matrix" sounds like a weapon from Marvel Universe.
If you're going to try something new, there's no better (worse) place to start than an expensive, targeted product used by the most exacting customers, on a highly complex core. If it works for a 72-core processor, it's probably okay on a 22-core or less processor.
Exactly. They used it in enterprise-grade systems and had a year for further improving. It should be relatively problem-free compared to Zen.
 
Last edited:

Vasrass

New Member
Joined
Jun 16, 2017
Messages
2 (0.00/day)
Most important thing is missing:

When will it be available in Xeon/Skylake-X mobos and actually ship?

We don't need more paper launches and I doubt many here will buy Knights Landing for gaming.

If it was ready, Intel would have announced it now.

Second most important question:

Will this obsolete X299 platform before it is even out?
 
Joined
Feb 7, 2006
Messages
738 (0.11/day)
Location
Austin, TX
System Name WAZAAM!
Processor AMD Ryzen 3900x
Motherboard ASRock Fatal1ty X370 Pro Gaming
Cooling Kraken x62
Memory G.Skill 16GB 3200 MHz
Video Card(s) EVGA GeForce GTX 1070 8GB SC
Storage Micron 9200 Max
Display(s) Samsung 49" 5120x1440 120hz
Case Corsair 600D
Audio Device(s) Onboard - Bose Companion 2 Speakers
Power Supply CORSAIR Professional Series HX850
Keyboard Corsair K95 RGB
Software Windows 10 Pro
So if AMD's Infinity Fabric caused headaches for OSes and scheduling, how will this fare? At least with rings and CCXs it's easy to predict the penalty for going from Core-X to Core-Y. Here it'll be much more complicated as it will be dependant on how much other traffic is going on between cores.

Additionally, the nodes near the 'middle' of the mesh will have much more traffic going through them then any of the 'edge' nodes. When this scales large enough (100+) that effect will somewhat dissipate but for <20 cores it will probably be substantial.
 
Joined
Feb 12, 2015
Messages
1,104 (0.33/day)
Intel is really going with the full on Chinese approach to business, eh? :D

Their decisions sure are interesting. They could have done this years ago, but didn't. Now, it's all, "See, we have this stuff, too, look at us."

It must only be a matter of time (new architecture) until they direct copy and start slapping together MCMs, since yields are killing them on these large core counts.

There is no way this is just "ready". MCM's are so economical that Intel would have used them a DECADE ago if they could.


My guess is they started working on this tech a few years ago when they realized they would need it, and they still won't have it perfected to the level AMD has it for another 2 years at least.
 
Joined
May 7, 2009
Messages
5,392 (0.99/day)
Location
Carrollton, GA
System Name ODIN
Processor AMD Ryzen 7 5800X
Motherboard Gigabyte B550 Aorus Elite AX V2
Cooling Dark Rock 4
Memory G Skill RipjawsV F4 3600 Mhz C16
Video Card(s) MSI GeForce RTX 3080 Ventus 3X OC LHR
Storage Crucial 2 TB M.2 SSD :: WD Blue M.2 1TB SSD :: 1 TB WD Black VelociRaptor
Display(s) Dell S2716DG 27" 144 Hz G-SYNC
Case Fractal Meshify C
Audio Device(s) Onboard Audio
Power Supply Antec HCP 850 80+ Gold
Mouse Corsair M65
Keyboard Corsair K70 RGB Lux
Software Windows 10 Pro 64-bit
Benchmark Scores I don't benchmark.
So if AMD's Infinity Fabric caused headaches for OSes and scheduling, how will this fare? At least with rings and CCXs it's easy to predict the penalty for going from Core-X to Core-Y. Here it'll be much more complicated as it will be dependant on how much other traffic is going on between cores.

Additionally, the nodes near the 'middle' of the mesh will have much more traffic going through them then any of the 'edge' nodes. When this scales large enough (100+) that effect will somewhat dissipate but for <20 cores it will probably be substantial.

It didn't cause any issues with the scheduler. That was 1 or 2 Youtuber's who know nothing about OSes and schedulers blaming the first thing that came up in their "research. That is why there was never a patch from MS or AMD. AMD released a "patch" that was just a power management profile to adjust how Win10 handled lower power states.
 
Joined
Feb 7, 2006
Messages
738 (0.11/day)
Location
Austin, TX
System Name WAZAAM!
Processor AMD Ryzen 3900x
Motherboard ASRock Fatal1ty X370 Pro Gaming
Cooling Kraken x62
Memory G.Skill 16GB 3200 MHz
Video Card(s) EVGA GeForce GTX 1070 8GB SC
Storage Micron 9200 Max
Display(s) Samsung 49" 5120x1440 120hz
Case Corsair 600D
Audio Device(s) Onboard - Bose Companion 2 Speakers
Power Supply CORSAIR Professional Series HX850
Keyboard Corsair K95 RGB
Software Windows 10 Pro
It didn't cause any issues with the scheduler. That was 1 or 2 Youtuber's who know nothing about OSes and schedulers blaming the first thing that came up in their "research. That is why there was never a patch from MS or AMD. AMD released a "patch" that was just a power management profile to adjust how Win10 handled lower power states.

It required significant engineering effort to release the windows and linux patches that supported their NUMA topology. These patches were deployed long before Zen released (I think the Linux patches were in 2015?)

But support still need to be engineered and added to the OS.
 
Joined
Jun 28, 2016
Messages
3,595 (1.26/day)
So if AMD's Infinity Fabric caused headaches for OSes and scheduling, how will this fare?
Normally. It'll most likely "just work" - like Intel's stuff usually does, especially if it is first used in enterprise products.

Lets just repeat this once again, because it seems many of you didn't read the whole text (or previous comments).
You talk about this tech like if it was still being developed and a huge mystery. Mesh interconnect exists - Intel has already used it in their top CPUs. Now it's simply being implemented in cheaper models.
My guess is they started working on this tech a few years ago when they realized they would need it, and they still won't have it perfected to the level AMD has it for another 2 years at least.
Prepare to be surprised.
 
Joined
May 7, 2009
Messages
5,392 (0.99/day)
Location
Carrollton, GA
System Name ODIN
Processor AMD Ryzen 7 5800X
Motherboard Gigabyte B550 Aorus Elite AX V2
Cooling Dark Rock 4
Memory G Skill RipjawsV F4 3600 Mhz C16
Video Card(s) MSI GeForce RTX 3080 Ventus 3X OC LHR
Storage Crucial 2 TB M.2 SSD :: WD Blue M.2 1TB SSD :: 1 TB WD Black VelociRaptor
Display(s) Dell S2716DG 27" 144 Hz G-SYNC
Case Fractal Meshify C
Audio Device(s) Onboard Audio
Power Supply Antec HCP 850 80+ Gold
Mouse Corsair M65
Keyboard Corsair K70 RGB Lux
Software Windows 10 Pro 64-bit
Benchmark Scores I don't benchmark.
It required significant engineering effort to release the windows and linux patches that supported their NUMA topology. These patches were deployed long before Zen released (I think the Linux patches were in 2015?)

But support still need to be engineered and added to the OS.

Yeah. as far as I recall, NUMA support up to 64 cores was added to Windows 7/Server 2008 R2 long before Windows 10 was even a thing. <- that is why I said what I said. The "issue" Zen had was not related to the scheduler. That was never the problem.
 
Joined
Sep 15, 2011
Messages
6,467 (1.41/day)
Processor Intel® Core™ i7-13700K
Motherboard Gigabyte Z790 Aorus Elite AX
Cooling Noctua NH-D15
Memory 32GB(2x16) DDR5@6600MHz G-Skill Trident Z5
Video Card(s) ZOTAC GAMING GeForce RTX 3080 AMP Holo
Storage 2TB SK Platinum P41 SSD + 4TB SanDisk Ultra SSD + 500GB Samsung 840 EVO SSD
Display(s) Acer Predator X34 3440x1440@100Hz G-Sync
Case NZXT PHANTOM410-BK
Audio Device(s) Creative X-Fi Titanium PCIe
Power Supply Corsair 850W
Mouse Logitech Hero G502 SE
Software Windows 11 Pro - 64bit
Benchmark Scores 30FPS in NFS:Rivals
So basically it means that only 2 to 4 Cores can communicate at once?
 
Joined
Jun 28, 2016
Messages
3,595 (1.26/day)
So basically it means that only 2 to 4 Cores can communicate at once?
To be honest, it's always like that: only 2 cores can communicate at once. :p
In a mesh topology each node (core) communicates only with its neighbours. This solution uses a rectangular grid model, so each core has 4 neighbours.
 
Joined
Nov 2, 2016
Messages
65 (0.02/day)
Intel is really going with the full on Chinese approach to business, eh? :D

Their decisions sure are interesting. They could have done this years ago, but didn't. Now, it's all, "See, we have this stuff, too, look at us."

It must only be a matter of time (new architecture) until they direct copy and start slapping together MCMs, since yields are killing them on these large core counts.
To be fair they did MCM before AMD, as a last ditch attempt to beat AMD to market with a dual-core desktop CPU in Pentium D days (Smithfield).
 
Joined
Aug 18, 2013
Messages
27 (0.01/day)
Location
San Antonio, Texas
Processor Intel Core i7-4790K @ 4.5GHz
Memory G.SKILL Sniper 16GB DDR3 @ 2400MHz
Video Card(s) PNY GTX 1060 3GB @ 2GHz
Storage 2 x 128GB SSD RAID 0 / 3 x 1TB HDD RAID 0
Software Windows 10 Pro
There is an executive at Intel that has an undiagnosed condition somewhere...
 
Top