Intel's New 5th Gen "Emerald Rapids" Xeon Processors are Built with AI Acceleration in Every Core

[AleksandarK]

Today at the "AI Everywhere" event, Intel launched its 5th Gen Intel Xeon processors (code-named Emerald Rapids) that deliver increased performance per watt and lower total cost of ownership (TCO) across critical workloads for artificial intelligence, high performance computing (HPC), networking, storage, database and security. This launch marks the second Xeon family upgrade in less than a year, offering customers more compute and faster memory at the same power envelope as the previous generation. The processors are software- and platform-compatible with 4th Gen Intel Xeon processors, allowing customers to upgrade and maximize the longevity of infrastructure investments while reducing costs and carbon emissions.

"Designed for AI, our 5th Gen Intel Xeon processors provide greater performance to customers deploying AI capabilities across cloud, network and edge use cases. As a result of our long-standing work with customers, partners and the developer ecosystem, we're launching 5th Gen Intel Xeon on a proven foundation that will enable rapid adoption and scale at lower TCO." -Sandra Rivera, Intel executive vice president and general manager of Data Center and AI Group.




With AI acceleration in every core, 5th Gen Xeon processors address demanding end-to-end AI workloads before customers need to add discrete accelerators—including up to 42% higher inference performance and less than 100 millisecond latency on large language models (LLMs) under 20 billion parameters.

Compared with the previous generation of Xeon, these processors deliver 21% average performance gain for general compute performance and enable 36% higher average performance per watt across a range of customer workloads. Customers following a typical five-year refresh cycle and upgrading from even older generations can reduce their TCO by up to 77%.

Building on previous generations, 5th Gen Xeon processors bring new innovations to deliver performance and efficiency benefits to customers. 5th Gen Xeon processors support up to 64 cores per CPU and nearly 3x the maximum last-level cache from the previous generation. They offer eight channels of DDR5 per CPU, support DDR5 at up to 5,600 megatransfers per second (MT/s), and increase inter-socket bandwidth with Intel UPI 2.0, offering up to 20 gigatransfers per second (GT/s). Select 5th Gen Xeon processor-based instances will support CXL Type 3 workflows through leading cloud service providers (CSPs).

5th Gen Xeon processors are pin-compatible with the previous generation, and top original equipment manufacturers (OEMs), including Cisco, Dell, HPE, IEIT Systems, Lenovo, Super Micro Computer and others, will offer a broad selection of single- and dual-socket systems beginning in the first quarter of 2024. Major CSPs will announce availability of 5th Gen Xeon processor-based instances throughout the year.

Intel Trust Domain Extensions (Intel TDX) offers increased confidentiality and security at the virtual machine (VM) level, enhancing privacy and control over data. Introduced earlier this year with the 4th Gen Xeon processors and available from select CSPs, with the launch of 5th Gen Xeon processors, Intel TDX will be generally available to all OEM and CSP solution providers. Within an Intel TDX confidential VM, guest operating systems and VM applications are isolated from access by the cloud host, hypervisor and other VMs on the platform.

Intel's new processors provide impressive performance increases and overall benefits for the company's customers and partners and enable the growth of optimized performance across key workloads.

With the launch of the 4th Gen Xeon processors in January 2023 and the launch of 5th Gen Xeon processors today, Intel continues to execute on its roadmap commitments to customers. Looking ahead, future Intel Xeon processors coming next year will deliver major advances in power efficiency and performance. Sierra Forest, with E-core efficiency and up to 288 cores, will arrive in the first half of 2024 and Granite Rapids, with P-core performance, will closely follow.

View at TechPowerUp Main Site

 

[ncrs]

Linux benchmarks for those new Xeons have been posted on Phoronix.

(I'm not affiliated with Phoronix so I hope I'm not violating some anti-spam rule on TPU, but since there aren't many Linux and server hardware tests here maybe it's OK )

 

[Denver]

Linux benchmarks for those new Xeons have been posted on Phoronix.

(I'm not affiliated with Phoronix so I hope I'm not violating some anti-spam rule on TPU, but since there aren't many Linux and server hardware tests here maybe it's OK )

I don't wear glasses, but I may need to buy them; I may not be seeing properly. Does this show that the new Xeon is obliterated in 90% of benchmarks by EPYC Models released 1 year ago?

How will this handle Zen5 with its quadrillion of cores?

 

[ncrs]

I don't wear glasses, but I may need to buy them; I may not be seeing properly. Does this show that the new Xeon is obliterated in 90% of benchmarks by EPYC Models released 1 year ago?

How will this handle Zen5 with its quadrillion of cores?

Yeah, performance and efficiency do look disappointing, apart from benchmarks utilizing AMX instructions. They also have specialized accelerators like QAT used for encryption/hashing, but they are segmented and not available on every SKU. Again, special software support is required to take advantage of them, but if you can, then they provide superb performance and perf/power.
EPYCs can have more cores per package but are limited to dual CPU configurations while Xeons can do 2/4/8 sockets natively. I'd say it's a niche advantage.
EPYCs have more memory channels - 12 DDR5-4800 vs. 8 up to DDR5-5600 (because Intel segments memory speed from 4400 to 5600).
As far as I know Zen 5/5c are supposed to reuse the current SP5/SP6 platforms, so core counts aren't likely to increase.
Intel does have an advantage in manufacturing and potentially pricing since they aren't dependent on TSMC. They can "incentivize" adoption with hefty rebates

 

[Denver]

Yeah, performance and efficiency do look disappointing, apart from benchmarks utilizing AMX instructions. They also have specialized accelerators like QAT used for encryption/hashing, but they are segmented and not available on every SKU. Again, special software support is required to take advantage of them, but if you can, then they provide superb performance and perf/power.
EPYCs can have more cores per package but are limited to dual CPU configurations while Xeons can do 2/4/8 sockets natively. I'd say it's a niche advantage.
EPYCs have more memory channels - 12 DDR5-4800 vs. 8 up to DDR5-5600 (because Intel segments memory speed from 4400 to 5600).
As far as I know Zen 5/5c are supposed to reuse the current SP5/SP6 platforms, so core counts aren't likely to increase.
Intel does have an advantage in manufacturing and potentially pricing since they aren't dependent on TSMC. They can "incentivize" adoption with hefty rebates

Rumors as far as I remember pointed to EPYC Zen5 with up to 256 cores. This supposed advantage remains to be seen, AMD has been swimming for some time with its modular chip(chiplets) strategy.

 

[ncrs]

Rumors as far as I remember pointed to EPYC Zen5 with up to 256 cores. This supposed advantage remains to be seen, AMD has been swimming for some time with its modular chip(chiplets) strategy.

After some searching it looks like the leaks are suggesting 16 chiplets on the same socket as current EPYCs. That's a lot, not only physically but also the IO die would have to increase the count of Infinity Fabric links from current versions:

All that while allegedly using just 4nm TSMC that is already employed for Zen 4c in Bergamo? I'm skeptical, but will be pleasantly surprised if they manage to pull it off.

 

[Wirko]

(I'm not affiliated with Phoronix so I hope I'm not violating some anti-spam rule on TPU, but since there aren't many Linux and server hardware tests here maybe it's OK )

No worries, linking to Phoronix, Anandtech, Tom's, Videocardz, Igor's Lab, WC**tech and other sites is something we often do here. To Hexus and Tech Report, sadly, not.

After some searching it looks like the leaks are suggesting 16 chiplets on the same socket as current EPYCs. That's a lot, not only physically but also the IO die would have to increase the count of Infinity Fabric links from current versions:

A CPU package with 256 cores would have a severe deficiency of memory bandwidth. The JEDEC MRDIMM (multi-ranked DIMM) standard could overcome that but it's unclear if it's going to be introduced anytime soon, and the most recent mention of it (Anandtech) I can find is six months old.

 

[ncrs]

No worries, linking to Phoronix, Anandtech, Tom's, Videocardz, Igor's Lab, WC**tech and other sites is something we often do here. To Hexus and Tech Report, sadly, not.

Well the fact that there is a list of sites that are not welcome, and that you obfuscated one of the names makes my disclaimer a bit less paranoid

A CPU package with 256 cores would have a severe deficiency of memory bandwidth. The JEDEC MRDIMM (multi-ranked DIMM) standard could overcome that but it's unclear if it's going to be introduced anytime soon, and the most recent mention of it (Anandtech) I can find is six months old.

An excellent point. One could argue that 12 channels of DDR5-8000, which is defined officially by JEDEC, could be made to work in some workloads. Unfortunately we seem to be quite a bit away from achieving those numbers at scale.
As for MR-DIMM, unfortunately Intel seems to be going to do their own thing with MCR DIMMs, which is a similar technology. On the other hand STH's coverage of Computex has a bigger version of the photo also visible in the article you linked that says MR-DIMM will be compatible with Intel Granite Rapids. So there's hope that an official JEDEC standard will be available on both major x86 platforms.
The same article also suggests that AMD is about to move to 16 channel designs, but that's unlikely to happen for Zen 5.

 

[Wirko]

Well the fact that there is a list of sites that are not welcome, and that you obfuscated one of the names makes my disclaimer a bit less paranoid

To clarify, those 2 sites are not unwelcome, they're dead (for all we PC enthusiasts care). That's the sad part.

 

[Minus Infinity]

Can we please never again even allude to wcccraptech. This is the technology version of 8Chan. The commenters on that site are unhinged.

 

[Wirko]

Can we please never again even allude to wcccraptech. This is the technology version of 8Chan. The commenters on that site are unhinged.

Your observation is absolutely correct. But you're free to use your NI (natural intelligence) and stop reading at the last word of the article, like I do. Or delete <div id="comments"> from HTML, or have some monkey in Firefox do it for you.

Even TPU news writers sometimes post links to them. Here's their coverage of this Emerald Rapids news: https:/crapsomething.com/intel-5th-gen-xeon-cpus-official-emerald-rapids-up-to-64-cores-320-mb-cache-prices/

 

[Tomorrow]

Your observation is absolutely correct. But you're free to use your NI (natural intelligence) and stop reading at the last word of the article, like I do. Or delete <div id="comments"> from HTML, or have some monkey in Firefox do it for you.

Even TPU news writers sometimes post links to them. Here's their coverage of this Emerald Rapids news: https:/crapsomething.com/intel-5th-gen-xeon-cpus-official-emerald-rapids-up-to-64-cores-320-mb-cache-prices/

You know the funny part? Is you enable Enhanced Tracking Protection in the latest Firefox Nightly it gets rid of the comments too so a win-win.
But yes i agree that they put zero effort to moderating the comments there. I've made totally reasonable responses and the first reply to my comment start with the most vile profanity imaginable. Suffice to say i wont even bother responding at that point and block the user at that point. Comments relating to AMD/Nvidia/Intel are especially toxic there.