Mismatched Memory Speeds for upcoming CPU Reviews?

[chrcoluk]

In the past we've run all processors in CPU reviews at the same memory speed.

For DDR5, this was DDR5-6000 36-36-36-76, and people have complained that these speeds are not good.

So for this round I'm thinking, give every platform what works well with it, but still make it as fair as possible:

View attachment 388319
This would be DDR5-6000 CL30 scaled to other frequencies, while keeping the actual nanoseconds timings the same

I think this is slightly unfair to AMD though, because Intel gets higher frequency, and still same nanoseconds. On the other hand, AMD could have engineered their MC to be able to run higher speeds? But Intel could have engineered their CPU to work better with lower frequency?

How about this?

View attachment 388321

My view is the entire platform is effectively reviewed, so if the platform aka IMC allows better memory speeds, then it is tested that way, I wouldnt do manual overclock or voltage tuning (unless its to correct out of spec voltage), but rather just highest working XMP mode. Use the same DIMM for both platforms, but downclock when it isnt stable basically. It is either that or test only using JEDEC.

Rereading your post it seems we are actually on similar line of thought as usually multiple XMP profiles keep same timings and just adjust clocks anyway, like wise in my systems if the RAM is unstable, my first action is to keep same timings but reduce clock speed.

Essentially if you run it to work equally because of the weaker platform it becomes an unfair review for the stronger platform.

I was surprised it was so little. Once the review drops, you'll see all the other games too. Everything could be considered margin of error for fps. Though each game is run 3 times for a average.

The only areas I saw improvement is blender, Cinebench. Still it isn't enough make a major impact.

3DMark and AID64 are also distinctly different per memory. which doesn't really matter because it's synthetic benchmark. This would indicate that yes 6000 CL26 is Superior for X3D, but we need much more powerful video cards to see that advantage. I don't think the RTX 5090 is enough. Certainly not the RTX 4090.

If you have a 9800X3D, buy the cheapest ram you can find and spend more on the video card. I know that's not what memory vendors want me to say, but it's the truth.

Are you able to share everything you put on discord on to the forum? Some of us are not fans of discord, and I am not even sure if I am on the same discord server as you, the TPU discord I am on I dont see any of the staff members.

 

[alwayssts]

I think "mismatched but fair" is the only true way to go.

People buying those respective platforms should be aware of their quirks and buy accordingly.

If this is a low-speed but low-latency kit for AMD or a high-speed but higher-latency kit for Intel, this should not matter, as both are available and cater to each architecture.

As long as the settings equal out, I think it is not only right, and fair, but obligatory for a realistic scenario to show what each platform is capable of and catered toward (both in strengths/weaknesses).

 

[ir_cow]

Are you able to share everything you put on discord on to the forum? Some of us are not fans of discord, and I am not even sure if I am on the same discord server as you, the TPU discord I am on I dont see any of the staff members.

I wouldn't have shared if it was under NDA, but it is a upcoming review. I was just surprised that X3D really didn't care about the memory and you got great performance, which is why I posted.

 

[The King]

You can do a RAM scaling review on AMD and Intel and people can use that has a reference from the stock settings which you have already done.
You can go in depth with things like SR VS DR or higher MT vs lower MT etc.

A similar issue was also there when ZEN 3 was reviewed where some reviewers like Gamer's Nexus showed that Dual Rank RAM could increase gaming performance has much as 10% on ZEN 3 which can make the difference between winning and losing against the competition.

I voted that same RAM config should be used for both AMD and Intel.

 

[Assimilator]

Wow, you guys have been doing Intel dirty by testing CPU at the same memory speeds as AMD. When Intel CPUs can for 9000+ MT/s...

Aaaand here's the whining, exactly as predicted.

 

[Waldorf]

It should be mismatched, as long as its going for sweetspot (clock/timings/cost)>

 

[Bill_Bright]

Because AMDs and Intels use different motherboard/chipset platforms, it is impossible to have a truly level playing field for testing and evaluating. Therefore, maybe you should look at this differently and don't attempt to create an AMD vs Intel evaluation environment.

Instead, go for 2 distinct evaluation scenarios; one for AMD and one for Intel. Then reviews can be AMD vs AMD and Intel vs Intel. You can still use the same RAM, graphics, case, cooling, PSU, drives and testing/evaluation apps. But obviously the motherboards will still be different.

I think this will work because for many, if not most of us, we have already made our choice between Red or Blue. That is, we decide first if we are going AMD or Intel. Then we seek out which CPU and motherboard to get - based on the features we want and of course, our budgets. I know that's what I do. Same with graphics. I first decide if I want NVIDIA or AMD, then I go find a card that meets my needs and budget.

With two different testing/evaluation scenarios, you could tailor your AMD evals to optimize AMDs' advantages to make it easy for users to compare the AMD Ryzen 9 9950X3D with the Ryzen 9 9950X, for example.

Likewise, you could tailor your Intel reviews to optimize Intels' advantages to make it easy for users to compare the Intel Core i9 14901TE with the i9 14901E, for example.

If you then, later, on wish to have a "shoot-out" to compare the R9 9950X3D with the i9 14901TE, that could be a separate comparative review of each motherboard/CPU combo.

Of course, @Assimilator is right and someone will whine and complain that 2 distinct evaluations scenarios isn't any good. But to me, since I already know what brand CPU my next build will use, this will sure help me decide which processor to get. I don't think, I am alone on that.

When trying to do a direct Brand A to Brand B comparison, it is kinda like comparing the Honda Accord against a Toyota RAV4. One is a car, the other a SUV.

 

[Onasi]

When trying to do a direct Brand A to Brand B comparison, it is kinda like comparing the Honda Accord against a Toyota RAV4. One is a car, the other a SUV.

While the rest of your points have merit, even if I would personally heavily disagree, this is just a plain bad analogy. There is no functional difference between actual practical usage of Intel and AMD CPUs. They, if we talk about actual end-user experience, do the exact same thing. The reviews are an evaluation of which of them does that thing better. Obviously, there always be certain specific tasks in which one chip will be better, while another in which another wins, but creating an overall performance summary is still very much possible and desirable. If we talk flagship models, then it’s very much a fact that a 9950X is an overall more potent chip than the 285K. And, if one is looking for gaming performance, the 9800X3D will be objectively the beat currently possible choice, no if’s or but’s. Not because it’s an SUV to Intels Accord and is drastically different, but because it does this exact thing as any other CPU, just measurably better.
Anyway, I am rambling.

 

[Bill_Bright]

While the rest of your points have merit, even if I would personally heavily disagree, this is just a plain bad analogy.

Really? Criticizing the analogy for not being perfect is the point you want to make? It seems you just want to argue (whine @Assimilator?) - or go fanboy over AMD? Regardless, clearly you missed the point entirely.

There is no functional difference between actual practical usage of Intel and AMD CPUs.

And not to mention, you are just wrong. As W1z has correctly noted, there are functional differences which makes comparing the two platforms (because they MUST use different motherboards/chipsets) "fairly" impractical.

The point I was making, even if not a perfect analogy (please note I said "kinda like!) is even though both SUVs and cars can haul people and cargo from point A to point B, one is a sedan and the other is a SUV. It would be "kinda like" comparing a SUV with a pickup truck.

Sure, one can compare an SUV with a sedan. But the point is, the buyer probably already decided on one or the other and therefore, the better comparison would be an Accord vs Camry or RAV4 vs CRV. Sorry if that flew right over your head.

 

[A Computer Guy]

In the past we've run all processors in CPU reviews at the same memory speed.

For DDR5, this was DDR5-6000 36-36-36-76, and people have complained that these speeds are not good.

So for this round I'm thinking, give every platform what works well with it, but still make it as fair as possible:

View attachment 388319
This would be DDR5-6000 CL30 scaled to other frequencies, while keeping the actual nanoseconds timings the same

I think this is slightly unfair to AMD though, because Intel gets higher frequency, and still same nanoseconds. On the other hand, AMD could have engineered their MC to be able to run higher speeds? But Intel could have engineered their CPU to work better with lower frequency?

How about this?

View attachment 388321

If the RAM speeds are going to be mismatched then I would prefer to see RAM speeds typical or compatibility optimized for each CPU ( and matched against specs of midrange ram pricing at the time) since that is how people are generally going to use it and assemble their system. So for AM5 6000MT/s (sweet spot) and Intel whatever is typical for those.

@ir_cow
You want me to run 9800X3D and 14900K with DDR5-5600 ? and 7800X3D with DDR5-5200 ? That's stupid, there's one or two sites that do that and they keep getting shit from everyone

Perhaps you can do one test at the CPU spec speed so people can see what they are not missing? I know it will never happen but personally I would love it if some 3600MT/s ECC and 6000MT/s ECC make it on the list for comparison sake. If there isn't really much of a difference I might go for 128GB ECC again on AM5.

Nobody I know in the benching community runs 1:1 6000mt/s. Most of them 8000mt/s and higher 1:2.

Perhaps some deeper investigation 1:1 6000mt/s vs 1:2 8000mt/s+ on AMD. I'm willing to bet 1:2 big memory bandwidth would win

But many actually run their computers at 1:1 at a lower cost than 1:2 8000MT/s. Anyway I don't think there a way to make everyone happy on this issue. A decision will have to be made.

 

[Onasi]

Really? Criticizing the analogy for not being perfect is the point you want to make? It seems you just want to argue (whine @Assimilator?) - or go fanboy over AMD? Regardless, clearly you missed the point entirely.

Yes, I did not like the analogy and thought it inappropriate. Not sure why me commenting on that is verboten? And how you arrived at the conclusion that I intend to “fanboy” over AMD is something that I cannot possibly fathom.

And not to mention, you are just wrong. As W1z has correctly noted, there are functional differences which makes comparing the two platforms (because they MUST use different motherboards/chipsets) "fairly" impractical.

I will quote myself since you seem to have missed my point (ironically):

There is no functional difference between actual practical usage of Intel and AMD CPUs. They, if we talk about actual end-user experience, do the exact same thing.

Allow me to reiterate - the technical differences between platforms are really only of interest to us enthusiasts. For the average person reading the review they are utterly superfluous and matter not. It doesn’t really make any difference to them whether the comparison is fully “fair”, being close enough is good enough (which is what the thread is about - how close enough should we go before it skews another way in terms of “fairness”). All that matters in the end of the day for the consumer is which CPU in their intended price range is generally faster, be it overall or in some task they specifically are interested in, be it gaming, MT workloads like rendering, video editing, whatever it may be. That is what gives the reviews their worth - comparative analysis. Not evaluating Intel products against AMD products is, even if “fair-er”, is not particularly useful for the end-reader. I don’t think W1zz (or any other PC hardware journalist) would ever even consider going that route.

 

[ShrimpBrime]

But many actually run their computers at 1:1 at a lower cost than 1:2 8000MT/s. Anyway I don't think there a way to make everyone happy on this issue. A decision will have to be made.

I agree. Users do all sorts of configurations.

We are just talking about trying other speeds than the usual 6000mt/s.

The rest is all semantics. AMD does this, Intel does that. He turned on XMP. That user bought a pre-built and wouldn't know what XMP was if it hit him in the face. That user over there knows all about hard drives. That user sticks to laptops.

So forth.

I think some other testing besides 6000mt/s for everything is good. Do a review or two, doesn't work out, go back to just 6000mt/s on everything. No big deal.

 

[mb194dc]

Do nothing, all the same speed.

Then write a separate article comparing the maximum speeds obtainable on each platform with the top CPU and showing price / performance of more expensive ram.

All bases then covered

 

[Frick]

on 1080p..on higher resolutions you're hardly making progress by spending more on X3D as well..I have a test bench with a 9700x and a 9800x3D and both are just within a margin for 3-5fps apart on 2k and its even gone on 4k..

Hugely depends on what you're playing. Link. (MS Flight Sim 2020)

 

[thestryker6]

To all those who think using the same speed makes sense:

Does it make sense that TPU reviewed ARL with 6000 when the CPU officially supports 6400 out of the box?

Does it make sense to run DRAM at a not officially supported speed/latency which benefits one company over the other?

Does it make sense going forward to be locked in to a specific speed when the next CPUs (Intel for sure and AMD very likely) also officially support higher speeds than being used for testing?

 

[ShrimpBrime]

To all those who think using the same speed makes sense:

Does it make sense that TPU reviewed ARL with 6000 when the CPU officially supports 6400 out of the box?

Does it make sense to run DRAM at a not officially supported speed/latency which benefits one company over the other?

Does it make sense going forward to be locked in to a specific speed when the next CPUs (Intel for sure and AMD very likely) also officially support higher speeds than being used for testing?

It's a review site.

What would you suggest to increase your views and popularity?

Maybe get into acting like Linus and Steve? Perhaps drop a video card on the floor in your video?

Or maybe just try some different ram speeds instead?

 

[Assimilator]

To all those who think using the same speed makes sense:

Does it make sense that TPU reviewed ARL with 6000 when the CPU officially supports 6400 out of the box?

Does it make sense to run DRAM at a not officially supported speed/latency which benefits one company over the other?

Does it make sense going forward to be locked in to a specific speed when the next CPUs (Intel for sure and AMD very likely) also officially support higher speeds than being used for testing?

Whining continues.

 

[R-T-B]

Whining continues.

You called it but tbf anyone who knows the history of threads like this knew it was a sure thing.

 

[Assimilator]

You called it but tbf anyone who knows the history of threads like this knew it was a sure thing.

Which is exactly why I don't understand why W1zz even bothered making this thread. He knows that whatever path he chooses is going to have people complaining, and as such only sane thing to do is just tell the complainers to bugger off somewhere else if they have such a problem with how he chooses to perform his reviews. There is a time and a place to consider the wants and needs of the community, and this ain't one of them.

 

[Bill_Bright]

There is a time and a place to consider the wants and needs of the community, and this ain't one of them.

One of, if not the most basic rules of public speaking is to "know your audience". Perhaps what needs to be done is to define what "community" means in this sense.

Does it mean the general "global" population - e.g. "normal users"? Or does it mean the much smaller (niche) community of TPU enthusiasts/regulars?

I personally find the PSU reviews invaluable since, sadly, many PSUs don't meet published specs or ATX standards. But I find CPU reviews less useful - except for meeting specs, stability issues, etc. I just am not interested in getting a few more Hertz or FPS out of my hardware. But then that's me.

 

[thestryker6]

Whining continues.

What a wonderful, useful, addition to the discussion this entire thread was made for. Perhaps if you engaged your brain instead of whatever this stupidity is you'd see the obvious point being made: using a single, arbitrary, DRAM speed doesn't make sense.

 

[Assimilator]

What a wonderful, useful, addition to the discussion this entire thread was made for. Perhaps if you engaged your brain instead of whatever this stupidity is you'd see the obvious point being made: using a single, arbitrary, DRAM speed doesn't make sense.

Rich, coming from someone who made a post that hasn't covered anything that hasn't been added by previous posts.

 

[thestryker6]

Rich, coming from someone who made a post that hasn't covered anything that hasn't been added by previous posts.

Oh? Someone in this thread pointed out TPU tested ARL at 6000 when it officially supports 6400? Someone also mentioned CPUs going forward officially supporting higher memory speeds than being suggested here? Looks like only one of the two of us is capable of reading before typing.

Since critical thinking is lost on you the point behind my post was to reframe the question being asked here to get people who think a single arbitrary speed is a good choice to understand why it isn't. I already gave my suggestion as to the way testing should be done going forward so didn't feel the need to reiterate that, but I didn't address why a single arbitrary speed was not a good choice.

You know what I find especially funny? All you've added to this thread is whining about people who have opinions. I'd say ones that were different from yours but you've not added your own opinion at all instead opting to whine and whine some more.

 

[DudeBeFishing]

I don't care what you do, just as long as you're transparent about it. No matter what you do, there will be some variable that can be considered unfair.

I'd prefer if more of the RAM timings were listed, like trfc, trrd, tfaw, etc, but that will confuse viewers more than help them. I doubt even %1 of this forum know what timings each memory IC and brand can safely run at.

 

[RootinTootinPootin]

Hugely depends on what you're playing. Link. (MS Flight Sim 2020)

View attachment 388569

I don't play that gamer specifically, the only instance to benefit from an X3D on higher reso's are games with lobbies upto 100+ players where everything and anything can go random..but not by much if you'd ask me (maybe could depend again on what you're playing), I'm as well hardly convinced on that chart you're referring..(ditto)