Thursday, October 19th 2017

AMD Expresses its Displeasure Over Intel's PT Benchmarks for 9th Gen Core

AMD gave its first major reaction to the Principled Technologies (PT) controversy, in which it came out strongly against the questionable methods PT employed, in its performance comparison between the Core i9-9900K and AMD Ryzen 7 2700X, in addition to certain other Ryzen Threadripper series products. In its response, AMD made its official position on controversy clear - it is not happy with PT.

AMD prepared a long list of flaws with PT's original testing, and the areas where it did not correct the mistakes in its second testing. The company also put out a list of its own "best practices" for comparative benchmarking, which prescribes "sanitizing the operating system," "sanitizing the platform" for stock vs. overclocked testing, "sanitizing the data," and to not create a vast disconnect between the test environment and the real-world.
Add your own comment

26 Comments on AMD Expresses its Displeasure Over Intel's PT Benchmarks for 9th Gen Core

#1
atomicus
Their orginal methodologies were so flawed it really begs the question how they've surivived this long as a company. Such basic inept errors were made, it would surely call in to question all their previous work over the years. If I were a client of theirs I'd certainly be looking elsewhere. It's beyond farce.
Posted on Reply
#2
R0H1T
Yeah no doubt about it, it's like the conclusions were preordained, sadly such practices are far too common & swept under the rug with the excuse that everyone does it :shadedshu:
Posted on Reply
#4
HTC
I'd like if AMD would, with whatever the next flagship CPU it launches would make 2 separate testings:

- the right way, as described in the 2nd pic of the OP
- the wrong way, as made in version one of PT's review but, in this occasion, favoring AMD instead of Intel, obviously

Would be interesting to see the comparison ...
Posted on Reply
#5
Prince Valiant
HTC said:
I'd like if AMD would, with whatever the next flagship CPU it launches would make 2 separate testings:

- the right way, as described in the 2nd pic of the OP
- the wrong way, as made in version one of PT's review but, in this occasion, favoring AMD instead of Intel, obviously

Would be interesting to see the comparison ...
I'd like to think AMD's PR team is intelligent enough not to make themselves hypocrites.
Posted on Reply
#6
Casecutter
Wow ... Spectre, cheating, and running hot like a new i9 9900K even though Intel gave them solder!
The show must go on...
Posted on Reply
#7
Lightning
Who cares about some rando website ? TPU review is up.:lovetpu:
Posted on Reply
#8
Ubersonic
"Enclosed thermal environment unsuitable for comparative testing" - What the systems had cases as opposed to open rigs? IMO that's a plus as it's mimics real world use..


Testing between AMD/Intel CPUs was so much easier when they used the same socket/platform lol, such a shame Intel had to block that >.>
Posted on Reply
#9
eidairaman1
The Exiled Airman
Intel, of course bribing.

AMD should have people doing a slew of tests live with specs all posted on Youtube.
Obvious they can't trust PT for their unprofessional style.
Posted on Reply
#10
Darmok N Jalad
PT was used to give Intel an out. It’s not false advertising, as an independent company did the testing. Everyone will quickly forget the company “Principled Technologies,” just like they did with CTS-Lab, a company that still hasn’t published the documents on the AMD flaws it found, and it has done no other research since.
Posted on Reply
#11
jsjones008
Ubersonic said:
"Enclosed thermal environment unsuitable for comparative testing" - What the systems had cases as opposed to open rigs? IMO that's a plus as it's mimics real world use..


Testing between AMD/Intel CPUs was so much easier when they used the same socket/platform lol, such a shame Intel had to block that >.>
I think they're referring more to the specific case PT was using. The case they were using has very little direct airflow, and they were using the stock downdraft cooler on the AMD benches. This created a scenario where the CPU was not being properly cooled for the setup.
Posted on Reply
#12
cdawall
where the hell are my stars
I mean amd isn't exactly the king of fair in pr events on their own side, but this webpages delivery was a trashcan fire.
Posted on Reply
#13
jsjones008
Darmok N Jalad said:
PT was used to give Intel an out. It’s not false advertising, as an independent company did the testing. Everyone will quickly forget the company “Principled Technologies,” just like they did with CTS-Lab, a company that still hasn’t published the documents on the AMD flaws it found, and it has done no other research since.
The thing with PT is like you said, they were hired by Intel.

My curiosity on the matter is if PT's original testing was carried out as Intel asked, purposefully giving them the numbers they wanted, and then the follow up was OK'd by Intel to try and stop the bleeding.

I know Steve from Gamer's Nexus interviewed one of the founders of PT, and according to him they tested it how they wanted and how they thought was appropriate, but if a big corporation is paying you big money for something, you're not going to throw said corporation under the bus.
Posted on Reply
#14
Upgrayedd
Still tho "Gaming Mode" .. seriously..you call it game mode then get pissed when they use it for games? Why even have it then?
Posted on Reply
#15
opteron
Slizzo said:
Go get 'em AMD.

And to think, AMD has contracted PT in the past to do testing on their stuff as well...

https://www.principledtechnologies.com/portfolio-marketing.php#AMD
Well, $$$ talks louder. PT will surely get banned by AMD for sure, given the high fees $$$ paid by Intel.


"sanitizing the operating system," "sanitizing the platform" for stock vs. overclocked testing, "sanitizing the data,"

let's santize PT...
Posted on Reply
#16
moproblems99
Upgrayedd said:
Still tho "Gaming Mode" .. seriously..you call it game mode then get pissed when they use it for games? Why even have it then?
Game Mode is for Threadripper. Why AMD gave it to Ryzen, who knows.
Posted on Reply
#17
crazyeyesreaper
Chief Broken Rig
Upgrayedd said:
Still tho "Gaming Mode" .. seriously..you call it game mode then get pissed when they use it for games? Why even have it then?
ITs because the game mode is meant for legacy titles. Some games won't even run if more than 4 cores are present or have issues. Supreme Commander comes to mind.
Posted on Reply
#18
Vayra86
Well constructed response. I hope it gets enough exposure...
Posted on Reply
#20
Fluffmeister
Why get upset anyway? This duopoly can't fail whatever the other party does.
Posted on Reply
#21
Aquinus
Resident Wat-man
Fluffmeister said:
Why get upset anyway? This duopoly can't fail whatever the other party does.
So long as everyone is content to rely on x86 for running their applications, it's likely going to remain that way. It's not like there are really any good options, even if the software support is there (like with a lot of ARM variants.)
Posted on Reply
#22
Pewzor
jsjones008 said:
I think they're referring more to the specific case PT was using. The case they were using has very little direct airflow, and they were using the stock downdraft cooler on the AMD benches. This created a scenario where the CPU was not being properly cooled for the setup.
While Lyle trying to blow the dust off the running 2700x bench using a heat gun during benchmarking session.

HTC said:
They don't need PT for that: remember Polaris launch, where they claimed 2 cards surpassed the 1080 in AotS @ a much lower cost?

How did that turn out?
You mean Raja who said it... and you know how that turned out? Intel hired Raja with offer he can't refuse.
Says a lot about Intel eh.
Raja probably botched Vega because Intel told him to do that, and if he do it right Intel will cut him a fat check and give him a management position so Intel can take on much weaker GPU maker AMD before taking on Nvidia...
Everything makes sense now.
Posted on Reply
#23
uuuaaaaaa
This speaks for itself:

Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit https://www.intel.com/benchmarks.

Intel is a sponsor and member of the BenchmarkXPRT Development Community, and was the major developer of the XPRT family of benchmarks. Principled Technologies is the publisher of the XPRT family of benchmarks.

Source:
https://www.intel.com/content/www/us/en/benchmarks/intel-product-performance.html
Posted on Reply
#24
HTC
Pewzor said:
You mean Raja who said it... and you know how that turned out? Intel hired Raja with offer he can't refuse.
Says a lot about Intel eh.
Raja probably botched Vega because Intel told him to do that, and if he do it right Intel will cut him a fat check and give him a management position so Intel can take on much weaker GPU maker AMD before taking on Nvidia...
Everything makes sense now.
Apparently, Raja was deprived of 2 / 3 of the engineers assigned to work on Vega early during Vega's development, which were re-assigned to work with one of CPU projects (dunno which Zen version), or so i've read: can't seem to find where i read that, atm. This is the reason Vega took way way longer to be introduced VS the planned time and that may, quite possibly, have a direct impact in it's current performance.
Posted on Reply
#25
OrumusST
Those benchmarks were obviously flawed and one can't help but wonder if it was done on purpose considering the circumstances but.....why? Intel was going to win. While Ryzen has taken AMD leaps and bounds it still is not as fast as Intel in most gaming situations...we know this. They know this...Intel knows this...So why try to make it seem like Intel is even further ahead then they are when you know the backlash is going to beat you up when the embargo is over? It makes no sense unless PT is truly that incompetent. As the saying goes ; Never assume malice when incompetence could be the motive.
Posted on Reply
Add your own comment