• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Principled Technologies Comments on their Intel Processor Study

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
18,262 (3.46/day)
Likes
20,007
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#1
Today, we have seen several reports that suggested Principled Technologies (PT) published misleading information in our recent study comparing Intel's gaming processors to AMD's. We apologize for our delay in responding, but it's been a busy day, and we wanted to be as thorough as possible in addressing inquiries concerning our testing. We'll address specific questions and share more detail on our methodology in a moment, but we first must respond directly to attempts to call our integrity into question.

For almost 16 years, we have tested products for our clients because they trust our integrity. We have worked not just for any one company but for dozens of the leading technology firms, including rivals such as Intel and AMD, Microsoft and Google, Dell and HP, and many others.



Those clients trust PT in part because our integrity and our technical knowledge are beyond reproach. We work hard to be the best in both of those areas. We chose our company name to emphasize our commitments to both technology and our principles. (And, accusers saying we are only in this for the money obviously haven't read our book, Limit Your Greed!)

Before going further, we thus must categorically deny any dishonesty in our work on this project for Intel or in any of our other projects.

Now that we've gotten that off our chests, let's address the specific questions from recent videos as well as subsequent posts and tweets.

Project overview
An overview of the project will provide useful context.

Our overall goal - and Intel's specific request for this project - was to create as level a playing field as possible for comparing the AMD and Intel processors as the majority of the gaming market would likely use them. To do that, we built and configured 16 systems for this comparative testing; we had two of each processor/motherboard configuration. We matched all components where possible, the only variances being the motherboards, CPUs, and CPU coolers. (Full details are in our interim report.)

In an effort to be very transparent, we published our interim summary report on Oct 8, 2018. We will continue to be transparent and responsive to any questions.

Responses to inquiries
We have received a number of inquiries regarding the testing methodology we used and the potential for bias in favor of Intel. We are providing additional information to be as transparent as possible and to help allay these concerns.

The following list summarizes many of the inquiries we have received and our responses. (We are continuing to work on addressing additional inquiries.)

  • Use of "Game Mode" on the AMD Ryzen 7 2700X: Some inquiries we have received concern the use of the Ryzen utility and the number of active cores in the AMD-based systems. Based on AMD's recommendations and our initial testing on the Threadripper processors, we found installing the AMD Ryzen Master utility and enabling the Game Mode increased most results. For consistency purposes, we did that for all AMD systems across Threadripper and Ryzen. We are now doing additional testing with the AMD systems in Creator Mode. We will update the report with the new results.
  • Cooler choice: We chose Noctua for the CPU coolers, due to having almost identical systems in the NH-U14S (Intel) and NH-U14S TR4-SP3 (AMD), which allowed us to maintain a comparable thermal profile. Because we were not performing any overclocking on any configuration, and because AMD has said it was a good cooler, we stuck with the stock AMD Ryzen 7 2700X Wraith Prism cooler.
  • Memory speeds: To have complete parity across all systems, and to allow the Intel Core i9 X-series and AMD Ryzen Threadripper to fully utilize memory bandwidth, we used four 16 GB DDR4 DIMMs on all configurations. We took the following memory configuration steps:

Intel
MSI Z390-A Pro motherboard (i9-9900K)
  • Load Optimized BIOS defaults
  • Enabled: Extreme Memory Profile (X.M.P.)
  • DRAM Frequency set to DDR4-2666
  • Asus Prime X299-Deluxe motherboard (i9-9900X ,i9-9980XE)
  • Load Optimized BIOS defaults
  • Enabled: Extreme Memory Profile (X.M.P.)
  • Disabled ASUS MultiCore Enhancement to use stock Intel multicore settings
  • DRAM Frequency set to DDR4-2666
  • Installed Intel Turbo Boost Maxdriver/utility

Asus Prime Z370-A (i7-8086K,i7-8700K)
  • Load Optimized BIOS defaults
  • Enabled: Extreme Memory Profile(X.M.P.)
  • Disabled ASUS MultiCore Enhancement to use stock Intel multicore settings
  • DRAM Frequency set to DDR4-2666
  • Power saving & Performance mode, set to Performance

AMD
Asus Prime X399-A (Threadripper 2990WX, Threadripper 2950X)
  • Load Optimized BIOS defaults
  • Verify that D.O.C.P is selected for AMD-equivalent memory settings to XMP
  • Performance Enhancer, set to Default
  • Disabled overclocking enhancement
  • DRAM frequency set to DDR4-2933
  • Set Core Performance Boost to Auto
  • Set performance bias to None
  • Installed Ryzen Master utility

Asus Prime X470 Pro (Ryzen 72700X)
  • Load Optimized BIOS defaults
  • Verify that D.O.C.P is selected for AMD-equivalent memory settings to XMP
  • DRAM frequency set to DDR4-2933
  • Set performance bias to None
  • Installed Ryzen Master utility

Resolution settings: One goal of this study was to test the CPUs and their graphics subsystems, not the GPUs, so we ran the tests at the most common gaming resolution (62.06%), 1920×1080, according to the Steam Hardware Survey: https://store.steampowered.com/hwsurvey/Steam-Hardware-Software- Survey-Welcome-to-Steam. This allowed us to minimize any GPU-based bottlenecks on the rendering pipeline.

Quality settings: We configured all games to use the "High" or equivalent preset, versus "Ultra" or other presets, also to emphasize CPU over GPU performance. In the case where there were only three presets, we chose the top preset.

Clarification of various installation questions: We installed all games using Steam or the Microsoft Store, and fully updated with the latest patches.

Motherboards: Re a Twitter comment from Cyber Cat @0xCats, "Hey @AMD Apparently according to @PrincipledTech @Intel is able to run Ryzen & Threadripper Chips on Z370 and Z390": Thanks for catching that copy/paste error in our configuration info. We made an error there. The correct processor/motherboard/BIOS version specs for the AMD procs we tested are the following: Ryzen 7 2700X/ASUS PRIME X470-PRO/4024 and Threadripper 2950X & 2990WX/ASUS PRIME X399-A/0807. We apologize for the error and will post a revised version (with changes noted) soon.

Because our goal is always to do the right thing and get the answers right, we are currently doing additional testing. We will share that data and will certainly call out if something is significantly different from what we've already published.

We are confident in our test methodology and results. We welcome questions and we are doing our best to respond to questions from our interim report, but doing so takes time. We will add responses if other issues come up.

Thanks for listening.
 
Joined
Sep 7, 2011
Messages
193 (0.07/day)
Likes
64
Location
Pekanbaru - Riau - Indonesia - Earth - Universe
System Name My Best Friend...
Processor Qualcomm Snapdragon 650
Motherboard Made By Xiaomi
Cooling Air and My Hands :)
Memory 3GB LPDDR3
Video Card(s) Adreno 510
Storage Sandisk 32GB SDHC Class 10
Display(s) 5.5" 1080p IPS BOE
Case Made By Xiaomi
Audio Device(s) Snapdragon ?
Power Supply 2A Adapter
Mouse On Screen
Keyboard On Screen
Software Android 6.0.1
Benchmark Scores 90339
#2
littlebit too late .. kudos to Steve from techspot.com ..

waiting for intel responses ..
 
Joined
Feb 23, 2008
Messages
579 (0.15/day)
Likes
142
Location
Montreal
System Name Sairikiki / Tesseract
Processor i7 920@3.56/ i5 4690k@4.2
Motherboard GB EX58-UDP4 / GB Z97MX-G5
Cooling H60 / LQ-310
Memory Corsair Something 12 / Corsair 16
Video Card(s) TriX 290 / Devil RX480
Storage Way too many...
Display(s) QNIX 1440p 96Hz / Sony w800b
Case AzzA 1000 / Carbide 240
Audio Device(s) Auzen Forte / board + Yamaha RX-V475 + Pioneer AJ
Power Supply Corsair HX750 / Dark Power PRO10
Software w10 64 / w10 64
Benchmark Scores I don't play benchmarks...
#4
A hilarious read, thank you. Tech sites/tube will pick this apart nicely.
 
Joined
Jul 18, 2012
Messages
353 (0.15/day)
Likes
153
Location
Germany
Processor i5-2500k
Motherboard Z68Pro3-M
Cooling HR-02 Macho
Memory 8 Gig
Video Card(s) RX580 Armor w/ Accelero Mono
Storage 850 EVO 512
Display(s) LG 27UM67 UHD
Case Don't believe in cases...
Power Supply SS G-650
#5
"We're sorry we got caught."
 
Joined
Mar 10, 2010
Messages
5,634 (1.79/day)
Likes
2,008
Location
Manchester uk
System Name Quad GT evo V ,Ttevo
Processor Amd R5 2600X@4.1 ,FX8350 @ 4.6ghz
Motherboard Crosshair hero7 ,Gigabyte 990X Gaming
Cooling 360EK extreme 360Tt rad all push/pull, cpu,NB/Vrm blocks all EK
Memory Corsair vengeance 16Gb 2933@16, @1600 cas8
Video Card(s) Rx vega 64 waterblockedEK,
Storage samsung 840(250OS), WD 1Tb+2Tb +3Tbgrn, 1tb seagate sshd hybrid(games)
Display(s) Samsung uea28"850R 4k freesync, LG 49" 4K 60hz ,Oculus
Case Lianli p0-11 dynamic ,Custom(modded) thermaltake Kandalf
Audio Device(s) Xfi creative 7.1 on board ,Yamaha dts av setup
Power Supply corsair 1200Hxi, Corsair ax1200
Mouse CM optane, no comment bad
Keyboard CM optane, asus but the worst free membrane one.
Software Win 10 Pro ,same
Benchmark Scores 15.69K best overall sandra so far 7703vega 3dmark timespy
#6
"clients trust PT in part because our integrity and our technical knowledge are beyond reproach"

And yet half the tech net question both hmn.
Also they made a fair few(wrong) assumptions considering their technical knowledge :p:D


And chat flip flopy crap, we went with this (res for eg) because everyone uses it , yeah guy everyone is using four sticks of ram ,wtaf.
 
Joined
Jan 8, 2017
Messages
3,099 (4.76/day)
Likes
2,253
System Name Good enough
Processor AMD Ryzen R7 1700X - 4.0 Ghz / 1.350V
Motherboard ASRock B450M Pro4
Cooling Scythe Katana 4 - 3x 120mm case fans
Memory 16GB - Corsair Vengeance LPX
Video Card(s) ASUS GTX 1060 Turbo 6GB ~ 2139 Mhz / 9.4 Gbps
Storage 1x Samsung 850 EVO 250GB , 1x Samsung 860 EVO 500GB
Display(s) 1080p TV
Case Zalman R1
Power Supply 500W
#7
How can/will anyone trusted PT
I for one have never heard of them. Doubt there was any trust to be lost in the first place in this particular area.
 
Joined
Nov 4, 2005
Messages
10,102 (2.13/day)
Likes
2,494
System Name MoFo 2
Processor AMD PhenomII 1100T @ 4.2Ghz
Motherboard Asus Crosshair IV
Cooling Swiftec 655 pump, Apogee GT,, MCR360mm Rad, 1/2 loop.
Memory 8GB DDR3-2133 @ 1900 8.9.9.24 1T
Video Card(s) HD7970 1250/1750
Storage Agility 3 SSD 6TB RAID 0 on RAID Card
Display(s) 46" 1080P Toshiba LCD
Case Rosewill R6A34-BK modded (thanks to MKmods)
Audio Device(s) ATI HDMI
Power Supply 750W PC Power & Cooling modded (thanks to MKmods)
Software A lot.
Benchmark Scores Its fast. Enough.
#8
They should post how much they got paid so AMD could have a fair shot at buying their morals as well, or maybe they could bid publicly so all would know how much it costs to buy them.
 
Joined
Dec 15, 2006
Messages
1,361 (0.31/day)
Likes
432
Location
Oshkosh, WI
System Name ChoreBoy
Processor 8700k
Motherboard Gigabyte Z 370 Gaming 7
Cooling Silverstone HE01
Memory CMK16GX4M2B3000C15
Video Card(s) EVGA 1080 SC
Storage 250GB Samsung 850 EVO and the trusty old 250GB Barracuda
Display(s) 32" 1080p TV and 22" Dell 1680x1050....
Case Fractal R6
Audio Device(s) On-Board
Power Supply 1000w Corsair
Software Win 10 Pro
Benchmark Scores A million on everything....
#9
Blowing smoke up corporate asses for sixteen years...... is more like it.
 
Joined
Oct 17, 2014
Messages
1,559 (1.06/day)
Likes
714
Location
USA
System Name $170 family PC, rest was reused old parts.
Processor Ryzen Athlon 200GE @ 3.2Ghz ($55)
Motherboard Biostar A320M Pro ($42)
Memory 8gb (2x4) DDR4 2666 CAS 15-15-15 ($63)
Video Card(s) Vega 3 Integrated
Storage 120GB SSD
Display(s) AOC 22V2H 21.5" Frameless 75hz Freesync
Case Thermaltake View 22
Audio Device(s) Schiit Modi 3 + Custom Tube Amp + Sennheiser HD58X
Power Supply Corsair 750w Bronze
Mouse Roccat Kone AIMO
Keyboard Logitech G610+ Cherry MX Red
Software Ubuntu
#11
If you watch Gamers Nexus Steve Burke interview PT co-founder from yesterday, the co-founder states that 64gb ram is considered normal for most users on 2700x and 9900k... lolol what a joke. yet, he has been benching computer stuff for 30+ years or so he claims... even if true, he is super out of touch with 2700k and 9900k/8700k owners.
 
Joined
May 30, 2018
Messages
371 (2.58/day)
Likes
250
Location
Florida
System Name getting back into it - constant WIP
Processor Ryzen 5 2600 [4.2GHz @ 1.3v]
Motherboard Asus ROG Strix X370-F
Cooling Scythe Mugen MAX - 2x 120mm NZXT stock fans top/rear exhaust - 2x 140mm Corsair ML140's front intake
Memory 2x8GB TridentZ RGB 3466Mhz CL16 [bad-luck b-die]
Video Card(s) MSI RX 580 Armor OC [1467MHz core / 1951MHz mem]
Storage Seagate Barracuda 1TB - Samsung 860 Evo 250GB
Display(s) crappy 720p 34" Vizio [for now - 1080p downscaling helps :P]
Case NZXT S340 Elite
Audio Device(s) Schiit Modi 2 Uber, Sys, Vali 2, Scarlett 2i2 2nd gen - LSR 305's, DT-990 Pro's, HD600's
Power Supply Corsair RM650x v2
Mouse iunno whatever cheap crap logitech *clutches Xbox 360 controller security blanket*
Keyboard see: mouse
Software Windows 10 Pro
Benchmark Scores Costs more than my last ex, but it never complains and has much better throughput.
#12
LOL, I mean they trusted Principled Technologies.

It's like handing your dog's leash to the alcoholic, crack-addicted homeless man outside while you go into the store to do your week's grocery shopping... ...can't really turn around and say it's his fault you lost your dog! Everybody in town knows what that guy's up to...

I mean, it goes deeper, too. Intel could've caught these issues if they simply looked over the methodology. That's where the real blame is. PT makes their methodology perfectly clear. It's not like they intentionally misled or fudged results. They provided their methods, and that's what they stuck to. And that's what Intel signed up for. Putting that stuff out there was their mistake. They hired the wrong people and there's no reason why they couldn't have figured that out beforehand. Somebody slipped up.

Like man, if they had simply looked into it they'd have known before any of this was even done. So now it looks like they went in looking for skewed results. And nobody will ever know. Maybe incompetence (probably,) but if it was intentional, the incompetence excuse still flies. Some heads hit the chopping block, life goes on, and a good number of people will never know and will still buy into it. Maybe it's not dishonesty, but the end result is the same...

Semi-related, I actually really respect the degree of detail to which PT delves into when outlining their methodology. They show you exactly how they got to where they did. Same can't be said for a whole lot of benchmarkers/reviewers. They open themselves up to a lot of criticism by showing their hand. Thus far I think they've handled it well, too. If they could get it together and unify on a sensible methodology, they'd be making some really meaningful contributions. And the thing is I think they want to, but they're out of touch. I really don't think they had bad intentions, not at the moment anyway.
 
Last edited:

cadaveca

My name is Dave
Staff member
Joined
Apr 10, 2006
Messages
17,050 (3.72/day)
Likes
11,457
#13
I think many of you are blowing this out of proportion, but yeah, there are some issues here.


My question is, how do I personally get paid to do these things for Intel? Forget all the other companies, how do I become the one that sets the standard these places are using, because this isn't the first time we've seen stuff like this.
 
Joined
Feb 18, 2012
Messages
1,404 (0.58/day)
Likes
1,114
System Name AORUS X7 DT v8
Processor i7-8850H
Cooling 2 laptop fans
Memory 16gb of 2666mhz DDR4
Video Card(s) Nvidia 1080
Storage 2tb 2.5 inch HD, 1tb SSD m.2
Display(s) 17.3" IPS FHD 1920x1080 144Hz
Power Supply 265w power supply
Mouse Logitech m705
Keyboard laptop keyboard
Software lots of movies and Windows 10 Pro with win 7 shell
Benchmark Scores High enough for me
#14
If you watch Gamers Nexus Steve Burke interview PT co-founder from yesterday, the co-founder states that 64gb ram is considered normal for most users on 2700x and 9900k... lolol what a joke. yet, he has been benching computer stuff for 30+ years or so he claims... even if true, he is super out of touch with 2700k and 9900k/8700k owners.
64gb??, I am still on 16gb and most people that I know in person still have 8gb. Only 2 that I know use 64gb and they are in the movie/song rendering business plus other things.
 
Joined
May 30, 2018
Messages
371 (2.58/day)
Likes
250
Location
Florida
System Name getting back into it - constant WIP
Processor Ryzen 5 2600 [4.2GHz @ 1.3v]
Motherboard Asus ROG Strix X370-F
Cooling Scythe Mugen MAX - 2x 120mm NZXT stock fans top/rear exhaust - 2x 140mm Corsair ML140's front intake
Memory 2x8GB TridentZ RGB 3466Mhz CL16 [bad-luck b-die]
Video Card(s) MSI RX 580 Armor OC [1467MHz core / 1951MHz mem]
Storage Seagate Barracuda 1TB - Samsung 860 Evo 250GB
Display(s) crappy 720p 34" Vizio [for now - 1080p downscaling helps :P]
Case NZXT S340 Elite
Audio Device(s) Schiit Modi 2 Uber, Sys, Vali 2, Scarlett 2i2 2nd gen - LSR 305's, DT-990 Pro's, HD600's
Power Supply Corsair RM650x v2
Mouse iunno whatever cheap crap logitech *clutches Xbox 360 controller security blanket*
Keyboard see: mouse
Software Windows 10 Pro
Benchmark Scores Costs more than my last ex, but it never complains and has much better throughput.
#15
I think many of you are blowing this out of proportion, but yeah, there are some issues here.


My question is, how do I personally get paid to do these things for Intel? Forget all the other companies, how do I become the one that sets the standard these places are using, because this isn't the first time we've seen stuff like this.
It's one benchmark among many... ...even if it is official, you still have to compare several to get a working picture. Most of them are going to get at least one thing wrong or have something not applicable to you.

And riight? The paid study racket sounds pretty sweet. But that's the thing... ...it's a paid study done for Intel. So when the results come out in their favor it's kinda like "Well, yeah...." I don't understand the call for pitchforks there. It pretty un-apologetically and transparently is what it is.

I think some stupid choices were made. I look at the mistakes, have a laugh, and move on. This outcome is pretty much par for the course on the internet these days. Not saying it's right, but it's predictable.
 
Joined
Jun 28, 2014
Messages
2,057 (1.31/day)
Likes
1,493
Location
Shenandoah Valley, Virginia USA
System Name Home Brewed
Processor i9-7900X and i7-7700K
Motherboard MSI X299 Gaming M7-ACK & ASRock Z270 Killer SLI/ac
Cooling Kraken X41 & Thermaltake Water 3.0
Memory 64GB DDR4-3000 GSKill RipJaws-V & 32GB DDR4-3200GSKill RipJaws-V
Video Card(s) 2X-GTX-1080 SLI & 2 GTX-1070 8GB G1 Gaming in SLI
Storage Both have 2TB HDDs for storage, 480GB SSDs for OS, and 240GB SSDs for Steam Games
Display(s) ACER 28" B286HK 4K & Samsung 32" 1080P
Case NZXT Source 540 & Rosewill Rise Chassis
Audio Device(s) onboard
Power Supply Corsair RM1000 & Corsair RM850
Mouse Generic
Keyboard Razer Blackwidow Tournament & Corsair K90
Software Win-10 Professional
Benchmark Scores yes
#16
Reviews from trusted sites are what I'm looking for.
I can wait for guaranteed-objective test results.

Considering the price of this CPU, objective testing is essential to me.
 

Ahhzz

Moderator
Staff member
Joined
Feb 27, 2008
Messages
4,749 (1.22/day)
Likes
4,234
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Corsair Vengeance K95
Software Win 7 Ult 64 bit
#17
If you watch Gamers Nexus Steve Burke interview PT co-founder from yesterday, the co-founder states that 64gb ram is considered normal for most users on 2700x and 9900k... lolol what a joke. yet, he has been benching computer stuff for 30+ years or so he claims... even if true, he is super out of touch with 2700k and 9900k/8700k owners.
seriously? I run 32gb, and bet I'm way outside the norm, or at least I was when I did that 3 years ago....
 
Joined
Jun 16, 2016
Messages
75 (0.09/day)
Likes
34
System Name Baxter
Processor Intel i7-5775C @ 4.0 GHz
Motherboard AsRock Z97-E ITX/AC
Cooling be Quiet! Shadow Rock LP with Noctua NF-A12 fan
Memory 16 GB 2400 MHz HyperX Savage DDR3
Video Card(s) MSI GTX 1080 Gaming X 8G
Storage Samsung 500 GB 850 Evo and 4TB Toshiba 7200 RPM HDD
Display(s) Vizio P65-F1 4KTV
Case Cooler Master Elite 130
Audio Device(s) Does my Intel iGPU count?
Power Supply Siverstone SX500-LG 500W
#18
The big issues I see with this testing:

- They "correctly" set the Threadrippers in Game Mode, on AMD's suggestion. This is fine. This will give the best results for Threadripper. THEN they set it for Ryzen for "consistency." That's not being consistent. That is enabling a Threadripper-exclusive option for no good reason. Clearly they had no idea what Game Mode was and they never checked the Task Manager to notice that Ryzen was being effectively cut off at the knees.

- They set the XMP profiles for every motherboard, and then manually changed the frequencies. If you know anything about XMP, it's that you should not just apply a profile and then change frequencies without changing the timings too. Just stick to XMP, or stick to the whitesheet for the timings. Doing half of both led to really loose memory timings. I also question why they would set Intel to 2666 and AMD to 2933. Apparently that is the max supported frequencies for those processors, but if you have XMP profiles (and I'm sure there were several different profiles), then just choose one and don't let the motherboard set the timings. Most people will do XMP and then STOP, or do aggressive memory timing tests to go beyond.

- Choosing to do 4 memory sticks on dual-channel setups was just adding too much uncertainty. They should have done 2x8 memory kits at highest supported XMP. Anything else was just asking for this kind of complex cock up.

- Their argument of just using the AMD cooler because AMD said their cooler is good is stupid. People choose coolers irrespective of their CPU choice. Not equalizing something like this is just lazy, and definitely leads to a situation where Intel is max cooled, and AMD is maybe not hitting the highest XFR bin. Really easy problem to avoid.

Overall, I think that this group probably did a couple starter tests to get their initial benchmark setup going, and went with the one that looked reasonable but gave Intel the biggest advantage. Because Intel was the customer to please. Or, maybe they did just end up with setup because they are dumb or old school or can't be asked to do the extra research that every good PCMR member does when they build a PC.
 
Joined
Apr 19, 2018
Messages
69 (0.37/day)
Likes
46
#19
I'm very surprised that Shrout did not get more of Intels crooked business. Probably next time...

He is very adept at using Intel made benchmarks on AMD hardware to showcase his sponsor...
 
Joined
Apr 1, 2008
Messages
3,327 (0.86/day)
Likes
909
System Name HTC's System
Processor Ryzen 5 1600
Motherboard Asrock Taichi X370
Cooling NH-C14
Memory G.Skill Kit 16GB DDR4 F4 - 3200 C16D - 16 GTZB
Video Card(s) Sapphire Nitro+ Radeon RX 480 OC 4 GB
Storage 1 Samsung NVMe 960 EVO 250 GB + 1 3.5" Seagate IronWolf Pro 6TB 7200RPM 256MB SATA III
Display(s) LG 27UD58
Case Corsair Obsidian 650D
Audio Device(s) Onboard
Power Supply Corsair TX750
Mouse Logitech Performance MX
Software Ubuntu 18.04 LTS
#20
Principled Technologies is sponsored by Intel and Intel is also a major developer of BenchmarkXPRT family of benchmarks, which is published by none other then Principled Technologies.

But did you know that AMD has commissioned not one but thirteen reports from them as well?
 

cadaveca

My name is Dave
Staff member
Joined
Apr 10, 2006
Messages
17,050 (3.72/day)
Likes
11,457
#21
The big issues I see with this testing:

- They "correctly" set the Threadrippers in Game Mode, on AMD's suggestion. This is fine. This will give the best results for Threadripper. THEN they set it for Ryzen for "consistency." That's not being consistent. That is enabling a Threadripper-exclusive option for no good reason. Clearly they had no idea what Game Mode was and they never checked the Task Manager to notice that Ryzen was being effectively cut off at the knees.
Game mode with "normal" Ryzen chips is offered because there are benefits to turning SMT off due to the cache design. If it had zero benefit, it wouldn't be offered. That's all that Game Mode did at first. Disable SMT. Then Threaderripper came out, and Game Mode disabled a whole core's worth of CPU, but leave the memory access intact. So can also set memory access to be local-only, but that disables half the memory.

Keep in mind, even offering game mode, shouldn't be done, by AMD, if it was truly detrimental.

- They set the XMP profiles for every motherboard, and then manually changed the frequencies. If you know anything about XMP, it's that you should not just apply a profile and then change frequencies without changing the timings too. Just stick to XMP, or stick to the whitesheet for the timings. Doing half of both led to really loose memory timings. I also question why they would set Intel to 2666 and AMD to 2933. Apparently that is the max supported frequencies for those processors, but if you have XMP profiles (and I'm sure there were several different profiles), then just choose one and don't let the motherboard set the timings. Most people will do XMP and then STOP, or do aggressive memory timing tests to go beyond.
You said it yourself, that's the max supported frequency as claimed by the CPU maker. It seems that what was tested was the maximum configuration for each CPU, without overclocking. That's actually quite fair.

- Choosing to do 4 memory sticks on dual-channel setups was just adding too much uncertainty. They should have done 2x8 memory kits at highest supported XMP. Anything else was just asking for this kind of complex cock up.
Again, maximum possible configuration. For me, 2x8 GB is not enough, and we need either 4x 4 GB or 4x 8 GB (filling all slots, with single-rank memory, is best). Maybe another memory benchmark analysis is in order here.

- Their argument of just using the AMD cooler because AMD said their cooler is good is stupid. People choose coolers irrespective of their CPU choice. Not equalizing something like this is just lazy, and definitely leads to a situation where Intel is max cooled, and AMD is maybe not hitting the highest XFR bin. Really easy problem to avoid.
Meh, you could be very right here, and I agree with this part 1000%

Overall, I think that this group probably did a couple starter tests to get their initial benchmark setup going, and went with the one that looked reasonable but gave Intel the biggest advantage. Because Intel was the customer to please. Or, maybe they did just end up with setup because they are dumb or old school or can't be asked to do the extra research that every good PCMR member does when they build a PC.
PCMR? Sigh. OK, throw yourself into a label, a stereotype, and think that's excellent. Do you have kids?

Keep in mind, I'm not really happy with this situation either, but man... what they did actually makes sense. Was it the perfectly optimized enthusiast configuration? Absolutely not. What they did was match maximum supported specifications only, and under those circumstances, AMD doesn't fare that well. Because the PC's enthusiast crowd is but a minor part of the market, doing things in this way really does make sense. It would have been nice to see overclocking performance tested as well though, and well.. AMD would look even worse then. I mean, that's what enthusiasts do, overclock, right?

Myself, I run a TR1950X in an ASRock board as my daily system. I have 7980XE and 7900X-based systems as well, but I don't use them often. I wanted to spend time with AMD's latest and see all sides since I had been using Intel for so long. There are benefits and negatives to both, and I think that this testing actually kind of showed that well.
 
Last edited:
Joined
Oct 11, 2018
Messages
7 (0.64/day)
Likes
3
#23
One does not simply say “we used the included box cooler because AMD SAID IT WAS A GOOD COOLER” and then in the next sentence pretend to be concerned with testing parity across systems.

How can you be so tone deaf? The entire purpose is to control for every possible variable, not every variable except the ones that AMD said we’re good.

I truly want to believe you you/them but your refusal to acknowledge that your methodology pales in comparison to part time youtuber’s, or that it’s a problem at all, makes that impossible.
 
Joined
Mar 10, 2015
Messages
541 (0.41/day)
Likes
354
System Name Wut?
Processor 4770K @ Stock
Motherboard MSI Z97 Gaming 7
Cooling Water
Memory 16GB DDR3 2400
Video Card(s) Vega 56
Storage Samsung 840 Pro 256GB
Display(s) 3440 x 1440
Case Thermaltake T81
Power Supply Seasonic 750 Watt Gold
#24
What they did was match maximum supported specifications only
It seems to be that you would want to try and test on the most likely scenario and not the maximum or minimum. Sort of like how the EPA tests fuel economy, they don't do it idling around town or pedal to the metal. They also appear to use a drive cycle that no one on Earth replicates with any frequency as people rarely achieve the EPA results.

You said it yourself, that's the max supported frequency as claimed by the CPU maker. It seems that what was tested was the maximum configuration for each CPU, without overclocking. That's actually quite fair.
It doesn't seem likely they are incompetent since they have 30 years of experience benchmarking so you would think that they would know how the memory systems of each architecture would work to avoid the disparity of timings. And going along with that, they seemed to tweak only the things that would negatively impact the Ryzen processors which leads to me to believe they are not incompetent and knew what they were doing.

All that said, I agree with you that it seems to be a mountain out of a mole hill. Just as you should never put all your eggs in one basket, you shouldn't rely on only a single source of information.
 
Joined
Oct 11, 2018
Messages
7 (0.64/day)
Likes
3
#25
You said it yourself, that's the max supported frequency as claimed by the CPU maker. It seems that what was tested was the maximum configuration for each CPU, without overclocking. That's actually quite fair.
They either had no idea what did what to what, or they did and that makes it even worse. as I said their methodology pales in comparison to part time amateur youtuber’s and there’s zero excuse for that. They contradict themselves claiming they understand the importance of parity and then saying but we used the stock cooler because “amd said it was good”. Come on.

As for RAM, they’re again trying to have it both ways and plead ignorance “but this is what the average guy would do” while also claiming to be professionals interested in parity across systems, but without actually taking the most basic steps of controlling for every possible variable and ensuring each system is presented in the best light possible, because nobody is interested in how a Ferrari 488 performs against a Ferrari 458 in limp mode because it’s throwing codes.

It seems to be that you would want to try and test on the most likely scenario and not the maximum or minimum. Sort of like how the EPA tests fuel economy, they don't do it idling around town or pedal to the metal. They also appear to use a drive cycle that no one on Earth replicates with any frequency as people rarely achieve the EPA results.

You don’t test “most likely” you test accurately. You test the performance potential of A vs. B, and you control for every possible variable, this is science 101, anything else is marketing at best.



It doesn't seem likely they are incompetent since they have 30 years of experience benchmarking so you would think that they would know how the memory systems of each architecture would work to avoid the disparity of timings. And going along with that, they seemed to tweak only the things that would negatively impact the Ryzen processors which leads to me to believe they are not incompetent and knew what they were doing.

All that said, I agree with you that it seems to be a mountain out of a mole hill. Just as you should never put all your eggs in one basket, you shouldn't rely on only a single source of information.
Does the data bring one to the conclusion that the i9 9900k is up to 50% faster “in the most likely scenario**”. No, and the EPA is a terrible reference, but they give highway miles/city mpg/and mixed, which is frankly amazing, because the EPA has actually painted a clearer picture than -anyone- and I never thought I would say that.

You don’t test “most likely” you test accurately. You test the performance potential of A vs. B, and you control for every possible variable, this is science 101, anything else is marketing at best.
 
Last edited:
Top