How can I achieve 128GB DDR5 memory with 19-13900KS

[Jamster]

My goal is to build a modest yet effective AI workstation featuring an Intel Core i9*13900KS processor, an ASUS ROG Maximus Z790 Extreme motherboard, and a Nvidia RTX 4090 graphics card.

I have trouble figuring out how to get 128GB RAM that is compatible. I read so much about DD5 and populating all four of the DIMM slots that I am truly scared to commit.

I hope I don't have to wait for a new processor with a Quad channel that maybe even fulfills the LGA 1700's 198 GB capability, Who knows when?

Needless to say, any assistance would be greatly appreciated...

 

[mstenholm]

There are three 4x32 GB on the qvl, all 4800 MT/s

 

[oxrufiioxo]

@ir_cow would probably be able to answer this pretty well

But like what @mstenholm says make sure you go with a set on the QVL.

I doubt quad channel will ever be a thing on LGA 1700 you might want to wait for the next Threadripper if you need more ram than 128GB.

 

[Jamster]

Well, that's not what I saw:

Vendors	Part No.	Size	XMP/EXPO Support	RAM Speed	Supported Speed	SS/DS	Chip Brand	Timing	Voltage	Socket Support
CORSAIR	CMT64GX5M2X5600C40(Ver5.43.13)	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-77	1.25	1,2
CORSAIR	CMT64GX5M2B5600C40(Ver5.43.13)	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-77	1.25	1,2
CORSAIR	CMT64GX5M2B5600C40(Ver.5.43.13)	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-77	1.25	1,2
G.SKILL	F5-5600J3636D32GX2-TZ5K	2x 32GB	XMP	5600	5600	DS	Samsung	36-36-36-89	1.25	1,2
G.SKILL	F5-5600J2834F32GX2-TZ5RS	2x 32GB	XMP	5600	5600	DS	SK Hynix	28-34-34-89	1.35	1,2
Kingston	KF556C40BBAK2-64	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-80	1.25	1,2
Kingston	KF556C40BBK2-64	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-80	1.25	1,2
G.SKILL	F5-6400J3239G32GX2-TZ5RS	2x 32GB	XMP	6400	6400	DS	SK Hynix	32-39-39-102	1.4	1,2
G.SKILL	F5-6400J3239G32GX2-TZ5RK	2x 32GB	XMP	6400	6400	DS	SK Hynix	32-39-39-102	1.4	1,2

Kingston	KF556C40BBK2-64	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-80	1.25	1,2
Kingston	KF556C40BBAK2-64	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-80	1.25	1,2

 

[oxrufiioxo]

Well, that's not what I saw:

Vendors	Part No.	Size	XMP/EXPO Support	RAM Speed	Supported Speed	SS/DS	Chip Brand	Timing	Voltage	Socket Support
CORSAIR	CMT64GX5M2X5600C40(Ver5.43.13)	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-77	1.25	1,2
CORSAIR	CMT64GX5M2B5600C40(Ver5.43.13)	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-77	1.25	1,2
CORSAIR	CMT64GX5M2B5600C40(Ver.5.43.13)	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-77	1.25	1,2
G.SKILL	F5-5600J3636D32GX2-TZ5K	2x 32GB	XMP	5600	5600	DS	Samsung	36-36-36-89	1.25	1,2
G.SKILL	F5-5600J2834F32GX2-TZ5RS	2x 32GB	XMP	5600	5600	DS	SK Hynix	28-34-34-89	1.35	1,2
Kingston	KF556C40BBAK2-64	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-80	1.25	1,2
Kingston	KF556C40BBK2-64	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-80	1.25	1,2
G.SKILL	F5-6400J3239G32GX2-TZ5RS	2x 32GB	XMP	6400	6400	DS	SK Hynix	32-39-39-102	1.4	1,2
G.SKILL	F5-6400J3239G32GX2-TZ5RK	2x 32GB	XMP	6400	6400	DS	SK Hynix	32-39-39-102	1.4	1,2

Kingston	KF556C40BBK2-64	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-80	1.25	1,2
Kingston	KF556C40BBAK2-64	2x 32GB	XMP	5600	5600	DS	SK Hynix	40-40-40-80	1.25	1,2

That's because asus makes it complicated on their site.... You have to look at a specific 32GB memory stick and see if it supports 4

 

[Jamster]

Is that to say they are also four-channel versions? But if so, that conflicts with the CPU's two-channel specification.

By the way, those RAMs are so unattractive...

 

[oxrufiioxo]

Is that to say they are also four-channel versions? But if so, that conflicts with the CPU's two-channel specification.

By the way, those RAMs are so unattractive...

Regardless of whether you run 2x32 or 4x32 it is still dual channel. LGA 1700 doesn't support quad channel.

As far as them being unattractive that is the price you pay to try and run 128GB of memory on a Z790.

 

[unwind-protect]

Is that to say they are also four-channel versions? But if so, that conflicts with the CPU's two-channel specification.

By the way, those RAMs are so unattractive...

What you need for your RAM is a good return policy. FWIW I bet it works fine if you just stuff random G.Skill in there.

Getting it to run at high frequency is a different matter. I would just get 5600 Mt/s and be done with it. Whatever speed you get after POST is what you get.

https://www.amazon.com/G-Skill-RipJaws-288-Pin-CL36-36-36-89-F5-5600J3636D32GA2-RS5K/dp/B09YK4QBWN/ref=sr_1_9?crid=33XF36BF7UEXH&keywords=g.skill%2B32%2Bgb%2Bddr5&qid=1682532421&sprefix=g.skill%2B32%2Bgb%2Bddr5%2Caps%2C179&sr=8-9&ufe=app_do%3Aamzn1.fos.18630bbb-fcbb-42f8-9767-857e17e03685&th=1

Returns accepted.

 

[ir_cow]

Another one of these threads..

Don't go with 128GB unless you absolutely need it. Intel 12/13th DDR5-4800 with 4x dual rank DIMMs is something not guaranteed to work. I believe it will default to JEDEC 4400 and often requires voltage tinkering to go up. Probably max would be 5200~

granted its been about 3 months since I last tied. Maybe some breakthrough in the BIOS department made things better.

Regardless of whether you run 2x32 or 4x32 it is still dual channel. LGA 1700 doesn't support quad channel.

Exactly, It will be still dual channel. Just with 4 DIMMs, two share the same data lanes

Edit: I do have 4x 24GB now that I should try. 96GB of single rank memory might fair much better.

 

[Jamster]

I apologize for lacking in originality with my predicament.

With smaller data sets, even a modest 32GB could be sufficient. However, I will need as much memory as possible for the huge data sets
I will be working with.

Initially, I had already chosen the new (4x48) 192GB Corsair RAM kit, but then I realized that 13900KS only supports up to 128GB!
I never considered this possibility after reading about the Z790 specs. That was a huge disappointment.

It seems like I have to choose between bread and butter... Butter being the speed! Which something I will need, while loading those huge data sets...

Asus has not got back to me yet. I think, in the end it comes down to how accommodating the BIOS is going to be...
G-Skill is also has not return to me yet... I've asked for advice and suggestions...

 

[ir_cow]

Can I suggest you go with Intel other platform that actually supports quad-channel?

It's gonna cost like triple, but that is what workstation / server platforms are for. Large amount of memory.

I'm 99% sure on this, that if the BIOS supports 48GB DIMMs (newest BIOS) that 4x48GB will work. However it is the same problem. Low speed wins the race.

 

[R0H1T]

Shouldn't the platform actually support up to 256GB at least when 64GB modules are released? Or can Intel/AMD limit this, if at all?

 

[Jamster]

It is apparent that Intel's decision not to fully support their own platform is merely a marketing tactic. It is speculated that their aim is to promote sales of Xeon processors. And then they can attach a stiff tag to the new desktop processors.

Currently, my financial plan limits me to a budget of $10,000. However, this sum is barely sufficient to acquire one of those 4th-generation scalable Xeon CPUs, which I am solely interested in purchasing. That was what I meant when I introduced my parts list: "A modest yet effective AI workstation." What you are saying about workstations is, of course, very true, I am hoping to be in a position, where I can build a system with a Xeon Platinum 8490H Processors, in a year or so. $17,000 for CPU, and $10,342.00 more for the Nvidia A100 card, now that would be a true workstation. Of course, at least another 10 to 20 grand more for storage and whatnot... If things go as planned. Fingers crossed...

For now, I will have to work with this humble budget of mine.

This is a quote from pcguide (best cpus for deep learning)

Of course after paying homage to where it belongs... "For demanding professional workloads, it is advisable to opt for AMD Threadripper or Intel Xeon W CPUs."

"The Intel Core i9-13900KS is widely regarded as one of the best CPUs for deep learning. Its processing power is so impressive that it can even rival AMD Threadripper CPUs, making it unnecessary to opt for one of those. One of the most significant advantages of the 13900KS is its 20 PCIe express lanes, which can increase even further with a Z690/Z790 motherboard. This is crucial since many deep learning tasks rely on the GPU, and the extra lanes provide more power for GPU acceleration. Its exceptional processing power, compatibility with deep learning libraries, and additional PCIe express lanes make it an ideal choice for deep learning tasks."

I am also considering getting a mediocre rig for the time being, working on cloud GPUs & CPUs, and looking out for a (4x48) CPU.

 

[unwind-protect]

No reason for any of this. Just get 128 GB from a good brand and try them out. If there is trouble try lower frequencies. It's absurd to go to a registered RAM platform for a mere 128 GB RAM.

 

[oxrufiioxo]

"The Intel Core i9-13900KS is widely regarded as one of the best CPUs for deep learning. Its processing power is so impressive that it can even rival AMD Threadripper CPUs, making it unnecessary to opt for one of those. One of the most significant advantages of the 13900KS is its 20 PCIe express lanes, which can increase even further with a Z690/Z790 motherboard. This is crucial since many deep learning tasks rely on the GPU, and the extra lanes provide more power for GPU acceleration. Its exceptional processing power, compatibility with deep learning libraries, and additional PCIe express lanes make it an ideal choice for deep learning tasks."

You'd think that having only 20 PCIe lanes would be it's biggest disadvantage not only can you not run more one GPU at X16 you are limited by how many drives you can have. Even on your Extreme you have listed if you use the top m.2 slot the second X16 slot is disabled.

 

[Outback Bronze]

Just did a search and I get 4x48GB for ROG Extreme. It's only the 13th gen K/KF (and assume KS) and not 12th gen that can apparently do it from their website.

ROG MAXIMUS Z790 EXTREME | Motherboards | ROG AU (asus.com)

Select 13th gen then scroll down on "size" tab to 4x48GB

TBH I think you'll be fine with any 2x32GB x2 from their website if you want to run 128GB.

 

[ir_cow]

Just did a search and I get 4x48GB for ROG Extreme. It's only the 13th gen K/KF (and assume KS) and not 12th gen that can apparently do it from their website.

Interesting. Good to know. I don't have any 2x48GB kits to try it out myself yet.

 

[Jamster]

You'd think that having only 20 PCIe lanes would be it's biggest disadvantage not only can you not run more one GPU at X16 you are limited by how many drives you can have. Even on your Extreme you have listed if you use the top m.2 slot the second X16 slot is disabled.

That's true. However, I wouldn't be able to run a second GPU card with RTX 4090 anyway.
And as far as extra drives go, I will have a LAN-connected external Drive bay to accommodate any additional drives. Although Lan also employs PCIe lanes, it could still provide some support... , I wish I could go with scalable Xeons and be free of all this shenanigans.

Just did a search and I get 4x48GB for ROG Extreme. It's only the 13th gen K/KF (and assume KS) and not 12th gen that can apparently do it from their website.
ROG MAXIMUS Z790 EXTREME | Motherboards | ROG AU (asus.com)
Select 13th gen then scroll down on "size" tab to 4x48GB
TBH I think you'll be fine with any 2x32GB x2 from their website if you want to run 128GB.

I could be more assured about this. I plan to purchase components in the US and construct the equipment abroad. Hence, encountering any issues may severely disrupt my schedule.

I'm familiar with the Ram kit you mentioned. Yes, the motherboard supports (4x48), however the Intel's CPU doesn't! I nearly purchased the RAM kit. After reviewing the z790 specifications, I was under the impression that the platform's new CPU could accommodate 192GB. And then I saw the 13900KS supports 128GB! Bummer!

I am grateful for all of your support. It's the only way to keep my mind from melting. Thank you.

 

[unwind-protect]

And then I saw the 13900KS supports 128GB! Bummer!

How do you know that? All I see is specs that might not have been updated with 48 GB modules became available.

 

[MHzTweaker]

Intel W790 chipset will get you where you need to go but OH MY is the price to play rather lofty.
The boards also need special DDR5 RAM too, but you can use the RAM capacities you want.

https://www.amazon.com/ASUS-Pro-WS-Motherboard-Server-Grade/dp/B0BZGGW2JP/ref=sr_1_2?crid=2R7OPH88SP64U&keywords=Asus+W790+motherboard&qid=1682696028&sprefix=asus+w790+motherboar%2Caps%2C234&sr=8-2&ufe=app_do%3Aamzn1.fos.ac578592-0362-4e0a-958c-0f2dd61d30d4

https://www.amazon.com/ASUS-Pro-WS-W790-ACE-Motherboard/dp/B0BZGM1P69/ref=sr_1_1?crid=2R7OPH88SP64U&keywords=Asus+W790+motherboard&qid=1682696071&sprefix=asus+w790+motherboar%2Caps%2C234&sr=8-1&ufe=app_do%3Aamzn1.fos.ac578592-0362-4e0a-958c-0f2dd61d30d4

These are "proper" workstation boards but also a new product, a different socket and RAM running new W2400 and W3400 series CPUs

Intel® Xeon® W-3400 and Intel® Xeon® W-2400 Processors Brief

Platform Brief: Platforms based on the Intel® Xeon® W-3400 and Intel® Xeon® W-2400 processors offer increased core counts for high-end compute needs.

www.intel.com

I understand this isn't what you asked for but is worth considering if you have a $10k budget. Certainly doable if you use their 16 core 32 thread CPUs

EDIT: link to CPU pricing etc....

Intel Launches Overclockable Xeon W CPUs up to 56 Cores: a Return to HEDT-Class Chips

They're pricey, but HEDT-class chips are back.

www.tomshardware.com

 

[Jamster]

How do you know that? All I see is specs that might not have been updated with 48 GB modules became available.

I am not sure what do you mean? CPU specs, or Z790 specs?

Intel W790 chipset will get you where you need to go but OH MY is the price to play rather lofty.
The boards also need special DDR5 RAM too, but you can use the RAM capacities you want.

I understand this isn't what you asked for but is worth considering if you have a $10k budget. Certainly doable if you use their 16 core 32 thread CPUs

EDIT: link to CPU pricing etc....

Intel Launches Overclockable Xeon W CPUs up to 56 Cores: a Return to HEDT-Class Chips

They're pricey, but HEDT-class chips are back.

www.tomshardware.com

Surely, this seems like a much more sensible solution. Although my previous research clearly suggests that, unless Xeons are the new scalable ones, the highly praised 13900KS (which I see you have too) is a much more viable choice for AI work. However, I will have to examine your suggestion further.

 

[xxabi]

Tried something similar also for AI training but bigger. My machine is an ASUS Z790 Hero, 13900KF (I think it's more stable for 24/365) with 128Gb Crucial Dominator Platinum DDR5 5200 and can't get this thing stable past 64Gb.

I've made some upgrades in the machine and found some extra problems and different approach for solutions:

Now I just run it with 64Gb RAM but I've put a RAID 0 2x8TB Nvme Corsair MP600. This gives theorically 16TB Virtual RAM that runs at 16gb/s so it's like an slow but usable RAM.

But... Windows 11 max virtual size is 256Gb per partition!!! I didn't found anything about this but it doesn't allow me to put more. So I have 2 partitions with total 512Tb of Virtual RAM and I will have to put extra partitions if I need more.

As storage, the machine uses 4x8TB Raid 0 Samsung SSD. For backup all of this a QNAP with 64GB RAM, a pair of 4x8TB HDD and also with a Quadro A2000 12Gb for inferencing using virtual machines.

Also, my GPU is a 4090 like yours but, using Thunderbolt capabilities of this motherboard, I have an external thunderbolt Nvidia 6000 Ada Generation with 48Gb, for using when 24Gb is not enaugh. Didn't find anyone using Thunderbolt in consumer mainboards for GPU and it works without any problem, so I can disconnect the external Thunderbolt card and connect to the QNAP whenever I want.

 

[Macro Device]

Even 10th gen supports 128 GB. Just get the four of any supported 32 GB RAM sticks and you'll be completely fine. Quad channel won't work but you don't need to bother much, quad channel is a luxury anyway.

CPU specs are launched without adjusting to the future so you don't need to be surprised if "suddenly" your CPU handles 2x or 4x the RAM it's "supposed" to handle.

Regarding the platform choice... I dunno, probably scalable Xeons are really so much better you should invest in them.

 

[phanbuey]

It depends on your motherboard and ram choice -- the processor will be fine.

If you're going gobs of ram I tend to stick to MSI boards - those BIOSES seem to have the best/easiest memory tuning on intel, and I've not had issues maxing out ram on them.

Gigabyte can be hit or miss, but i've not done a 128GB build on them yet.

The last workstation I built with an ASUS board was a disaster that would cycle every 5-8 days for no reason that we could find, swapped out the board and everything worked great. I personally don't trust ASUS bioses of late for a variety of reasons.

 

[AleXXX666]

How do you know that? All I see is specs that might not have been updated with 48 GB modules became available.

intel.com: Max Memory Size (dependent on memory type) 128 GB

It depends on your motherboard and ram choice -- the processor will be fine.

If you're going gobs of ram I tend to stick to MSI boards - those BIOSES seem to have the best/easiest memory tuning on intel, and I've not had issues maxing out ram on them.

Gigabyte can be hit or miss, but i've not done a 128GB build on them yet.

The last workstation I built with an ASUS board was a disaster that would cycle every 5-8 days for no reason that we could find, swapped out the board and everything worked great. I personally don't trust ASUS bioses of late for a variety of reasons.

and I got MSI H610 and RAM from QVL - I knew the MSI is "best" for RAM. And it didn't work lmfao. Everyone will be "best" if you get top-tier product, I once was having f**in' issues with RAM speed on Asus Z490P, but some pricier ASUS boards have no troulbles like that lol