AMD ditches sysmark 2012

Add them to nvidia and Via.

Why did AMD Quit?
According to a blog written by Nigel Dessau, Senior Vice President and Chief Marketing Officer of AMD, the problem with SYSmark is, that it runs a lot of demanding workloads that don't correspond with typical usage of most users:

"While SM2012 is marketed as rating performance using 18 applications and 390 measurements, the reality is that only 7 applications and less than 10 percent of the total measurements dominate the overall score. So a small class of operations across the entire benchmark influences the overall score.
In fact, a relatively large proportion of the SM2012 score is based on system performance rated during optical character recognition (OCR) and file compression activities - things an average user will rarely if ever do.
And SM2012 doesn’t represent the evolution of computer processing and how that evolution is influencing average users’ experience. SM2012 focuses only on the serial processing performance of the CPU, and virtually ignores the parallel processing performance of the GPU. In particular, SM2012 scores do not take into account GPU-accelerated applications that are widely used in today’s business environments."

Here is the follow up article which seems to present a dim view of things from inside AMD.

I don't know how objective bright side of the news is, but if these are legit comments from insiders, I'm a little worried.

[Melvis]

Fair enough i think.

[phanbuey]

Originally Posted by twilyth

Add them to nvidia and Via.



Here is the follow up article which seems to present a dim view of things from inside AMD.

I don't know how objective bright side of the news is, but if these are legit comments from insiders, I'm a little worried.

yeah its pretty obvious they dont like that the benchmark says they're slower lol... GPU accelarated "widely used" in business... that's just misleading. Are businesses using nvidia tesla? yes. Are they using AMD's gpus to crunch? probably not. And any workload that is crunched like that is very very specific, one would think this would be a custom written and highly developmental application, usually in it self representing a very small overall piece of the computing system even within the organization (i.e. R&D, finance etc). The company would still need run-of-the-mill servers to power its daily operations, and they would massively outnumber any GPU cluster.

[seronx]

Originally Posted by phanbuey

yeah its pretty obvious they dont like that the benchmark says they're slower lol... GPU accelarated "widely used" in business... that's just misleading. Are businesses using nvidia tesla? yes. Are they using AMD's gpus to crunch? probably not. And any workload that is crunched like that is very very specific, one would think this would be a custom written and highly developmental application, usually in it self representing a very small overall piece of the computing system even within the organization (i.e. R&D, finance etc). The company would still need run-of-the-mill servers to power its daily operations, and they would massively outnumber any GPU cluster.

Enterprise A pays Bapco for 1 license for Sysmark 2012: $10,000

Intel IGP, good enough pays $500k to upgrade to intel igp platform

THEN! they realized they have been had!

When most applications to run at high productivity with high-end gpus

• ABBYY® FineReader pro 10.0
• Adobe® Acrobat® Pro 9
• Adobe® After Effects® CS5
• Adobe® Dreamweaver® CS5
• Adobe® Photoshop® CS5 Extended
• Adobe® Premiere® Pro CS5
• Adobe® Flash® player 10.1
• AutoDesk® 3DS Max® 2011
• AutoDesk® AutoCAD® 2011
• Google Sketchup™ Pro 8
• Microsoft® Internet Explorer® 8
• Microsoft® Office® 2010
• Mozilla® Firefox® Installer
• Mozilla® Firefox® 3.6.8
• Winzip® Pro 14.5

Nvidia left first, mainly gpu
AMD left second, apu and amd gpus
VIA left third, chipset gpu is more powerful than intel igp

----
CPU performance had no relation to this event

[thunderising]

Bad bulldozer performance to blame

[seronx]

Originally Posted by thunderising

Bad bulldozer performance to blame

False

Ya, VIA and Nvidia left because of Bad Bulldozer performance

Originally Posted by seronx

False

I tend to agree. In the article they said they were testing ES's that they knew were crippled and were B0/1 silicon. That and the fact that they jumped on these quotes from presumed "insiders" so gleefully made wonder if this was really a genuine news piece or an attempt to make some of those people sitting on the fence waiting for this chip say screw it an just go with what Intel has to offer.

[seronx]

Originally Posted by twilyth

I tend to agree. In the article they said they were testing ES's that they knew were crippled and were B0/1 silicon. That and the fact that they jumped on these quotes from presumed "insiders" so gleefully made wonder if this was really a genuine news piece or an attempt to make some of those people sitting on the fence waiting for this chip say screw it an just go with what Intel has to offer.

Silicon wasn't crippled

BIOs are the only thing crippling Bulldozer

Unless you have the AMD AM3+ Test Bench Motherboard with the correct BIOs

You are doing it wrong...

So far only beta BIOs from Gigabyte and Asus make Bulldozer work but these test BIOs still have the FPU clock bug which will be fixed when the CPUs come out <-- someone else has said this

Originally Posted by seronx

Silicon wasn't crippled

BIOs are the only thing crippling Bulldozer

Unless you have the AMD AM3+ Test Bench Motherboard with the correct BIOs

You are doing it wrong...

So far only beta BIOs from Gigabyte and Asus make Bulldozer work but these test BIOs still have the FPU clock bug which will be fixed when the CPUs come out <-- someone else has said this

from the second article

After talking to journalists from other publications and developers that had access to engineering silicon, everyone was lead to believe that Bulldozer has a lot of "performance locks" which were set in place to prevent "premature benchmarculation" i.e. leaks

Not exactly an identified and credible source, but I assumed they were telling the truth about this much. Who know though.

[seronx]

Originally Posted by twilyth

from the second article

Not exactly an identified and credible source, but I assumed they were telling the truth about this much. Who know though.

It is the truth...

But it's not hard to figure out.....since the FPU is only half clocked



Real numbers are 14.029 to 15.029 but I am predicting 15 seconds
getting 15~ Seconds in SuperPI 1M @ 4.5GHz isn't that bad for a crippled Bulldozer



FPU should operate and work the same as Phenom IIs

but it can do double the rate of Phenom II but because the FPUs are half-clocked, you get weird leaks that don't match the Architecture

[MightyMission]

does disabling 5 cores help yield higher scores in sPi?

[seronx]

Originally Posted by MightyMission

does disabling 5 cores help yield higher scores in sPi?

No, SuperPi is single threaded

and it uses the x87 instruction set

[MightyMission]

i thought it was single threaded,so the core down is to get the highest oc possible right?

[seronx]

Originally Posted by MightyMission

i thought it was single threaded,so the core down is to get the highest oc possible right?

Yes, if you can drop 3(X4)-5(X6) cores and get a higher OC yes

Another more even handed article

Citing unfair and opaque testing methods that are not based on real-world computing models and software applications, AMD has resigned from the Business Applications Performance Corporation (BAPCo) and withdrawn its support for BAPCo’s SYSmark 2012 benchmark suite.

It all began a year ago, when Nigel Dessau, AMD’s Chief Marketing Officer, wrote a damning blog post about SYSmark’s poor and inacurate handling of heterogeneous computing. BAPCo promptly threatened to expel AMD, but didn’t follow through — and here we are, a year later, and AMD is leaving of its own accord. The exact reasons aren’t given, but it seems to mostly hinge on AMD’s shift towards APUs — combined CPUs and GPUs — and SYSmark’s inability to accurately measure their performance. SYSmark, in other words, is unfairly biased towards Intel’s raw processing power — and AMD can’t endorse a benchmark that ignores or downplays the strengths of its APUs. SYSmark’s bias is so strong, says Dessau, that it has caused governments to overspend around $8 billion on Intel chips.

The good news, though, is that AMD now wants to support an open and fair suite of processor benchmarks. AMD wants a benchmark that actually utilizes the A-Series’ on-chip GPU — and more importantly, AMD wants a benchmark that will show off its incoming GPGPU-like Fusion System Architecture. Rather nobly, AMD isn’t interested in making its own benchmark suite — instead, it wants to back the creation of a new industry consortium that will be tasked with creating an open benchmark to measure overall system performance.

Here's an article from ZDNet questioning the Brightside article

The latest dust-up in the AMD-versus-Intel never-ending conflict concerns BAPCo, a consortium of tech companies that releases a set of benchmarks, including, most importantly, SYSmark. This week, AMD quit the BAPCo board, and speculation over why has run rampant ever since.

Officially, AMD claims that the latest version of SYSmark, the just-released SYSmark 2012, fails to keep up with current computing trends and ignores the increasing role the GPU plays in computing tasks. Since AMD is trying to differentiate itself from Intel by boosting the GPU in its new chip designs, SYSmark’s reliance on just the CPU, in AMD’s opinion, doesn’t reflect everyday computing performance.

That’s the official word. But conspiracy theorists think there’s more to the story than just that. Most sensationally, Bright Side of News has run a piece with startling claims from “unnamed sources,” most notably that AMD decided to pull out of BAPCo because its forthcoming Bulldozer chips delivered underwhelming performance on SYSmark 2012, and that the company has spent resources toward surreptitiously undermining BAPCo through negative PR campaigns. According to the piece, AMD’s paranoia about SYSmark is related to the benchmark’s role in securing government contracts and the chip company’s fear that it won’t win new contracts with poor SYSmark 2012 results.

AMD supporters not only question the veracity of that report, but also point out that Nvidia and Via have recently quit the BAPCo board as well. That, fanboys argue, is proof that this isn’t just a case of sour grapes for AMD, but a sign that chip companies are tired of Intel’s dominance of the BAPCo benchmark-creation process. Those companies also have a stake in the GPU game, where Intel has a performance disadvantage. BAPCo detractors point to allegations that Intel has been able to write the code for BAPCo benchmarks in the past and requires BAPCo to use its code compiler as proof that AMD and other chip makers will never get a fair shake on SYSmark.

[Damn_Smooth]

Originally Posted by twilyth

Add them to nvidia and Via.



Here is the follow up article which seems to present a dim view of things from inside AMD.

I don't know how objective bright side of the news is, but if these are legit comments from insiders, I'm a little worried.

I think that the whole article is nonsense. Any AMD engineer is not going to go and publicly denounce the company, anonymously or not.

It makes more sense that a little known tech website would like to draw hits to it's site by starting controversy. If this so called engineer wanted to talk to someone, I doubt that he would do it to a website that is not that well known.

Even though it's already been said, I really don't think that Nvidia and VIA give a rat's ass about Bulldozers performance.

And one last thing, this "team" put together by AMD to discredit Sysmarck, is doing a horrible job. The only comments I have heard from AMD about it came from Nigel Dessau. He hardly constitutes a team, with one blog post explaining the situation. If AMD is employing the forces to do nothing, than hire me. I'm great at that.

[Widjaja]

I still do not 100% believe any of the articles I have read about Bulldozer performance.

At the moment the articles pretty much say these CPUs suck.

I mean who in their right minds hands out an engineering sample when they know the processor has bad performance and nothing is optimized for any significant scores?

That's like shooting yourself in the foot.

At the moment it's like saying hey this is the future of things to come from AMD CPUs.
Poor performance!
But it doesn't matter because it's cool to have 8 cores!

[seronx]

Originally Posted by Widjaja

I still do not 100% believe any of the articles I have read about Bulldozer performance.

At the moment the articles pretty much say these CPUs suck.

I mean who in their right minds hands out an engineering sample when they know the processor has bad performance and nothing is optimized for any significant scores?

That's like shooting yourself in the foot.

At the moment it's like saying hey this is the future of things to come from AMD CPUs.
Poor performance!
But it doesn't matter because it's cool to have 8 cores!

Do you buy Engineer Samples?

No, you buy Production samples....

Originally Posted by CompDracula

What is a Engineer Sample, Compared to a Production Sample!?!

To answer that^

Engineer samples should be considered defective goods

What we know/I heard around the web: AMD Bulldozer ES have:
Half-clocked FPU/Phenom II FPU(CPU/BIOs)
Locked Multiplier(CPU)
Higher Voltage(CPU/BIOs)
Higher Wattage(CPU)
Broken TurboCore(BIOs)
Lower Quality Silicon/Pilot silicon(FAB process)
more stuff just ask if you want more...

[Widjaja]

Originally Posted by seronx

Do you buy Engineer Samples?

No, you buy Production samples.....

Then what are these engineer samples supposed to represent?

It's the closest the general public has in terms of whats in store when it comes to the production model.

The results of the engineer samples are well....not great.
And what's the purpose of handing out "defective" CPUs to let people take look at the progress of whats to come unless you are aiming for bad press?

What else is the general public inclined to think?

I personally would want these new CPUs to be really good.
I would like to see AMD take the lead again.
But with the current articles and the engineering samples handed out by AMD, it's pretty hard not to see the future production model to be that much better.

One can only hope.

[v12dock]

Yield not performance

[seronx]

Originally Posted by Widjaja

Then what are these engineer samples supposed to represent?

It's the closest the general public has in terms of whats in store when it comes to the production model.

The results of the engineer samples are well....not great.
And what's the purpose of handing out "defective" CPUs to let people take look at the progress of whats to come unless you are aiming for bad press?

What else is the general public inclined to think?

I personally would want these new CPUs to be really good.
I would like to see AMD take the lead again.
But with the current articles and the engineering samples handed out by AMD, it's pretty hard not to see the future production model to be that much better.

One can only hope.

These Eng Samples aren't for the public...but for Mobos Manufactures to get their BIOses up to date

The other Eng Samples that show how good Bulldozer are locked away in a tight vault under the protection

You leak, we are putting you out


--------------
http://www.amdzone.com/phpbb3/viewto...f=532&t=138660

fun to read the fights

Originally Posted by scientia

AMD was ahead in SYSmark 98
So, BABco altered SYSmark 2000 to de-emphasize memory bandwidth and instead emphasize L2 cache speed. By an amazing coincidence AMD didn't have any L2 while Intel did.

AMD however caught up with Thunderbird which included L2. Nevertheless, AMD's processor had considerably more memory bandwidth. In other words, rather than just catching up, Thunderbird would have been far ahead of PIII if not for the bandwidth de-emphasis.

Then with SYSmark 2001, BABco decided to increase the emphasis on memory bandwidth.
By another amazing coincidence Intel had just released P4 with much higher memory bandwidth.

With SYSmark 2001 Office productivity AMD drops 41.8% while Intel only drops 36.4% with the same applications. BABco changed the weighting of the applications.

Then with SYSmark 2002, AMD drops 22.7% while Intel only drops 17% with the same applications. BABco again changed the weighting of the applications.

With SYSmark 2001 an Athlon XP 1800 would beat a P4 2.0 Ghz by 4%. However, with SYSmark 2002 the Athlon would lose by 8%.

So, from SYSmark 98 to SYSmark 2002 we can show that with each generation: Intel got better scores while AMD got worse scores. Yes, that's some coincidence.

More coincidences. With each new version, Anand defended the changes. But there was more to it. CSA Research made a competing product called OfficeBench. CSA had no ties to either Intel or AMD so theoretically it would be the most neutral. In 2002, Anand decided to drop OfficeBench. By an amazing coincidence, Intel's processors didn't do as well with OfficeBench. Another interesting example was using LightWave to render an image. Intel had a reccomended image. Yet, when Aces used a different image, Athlon XP went from being 20% slower to 5% faster.

AMD made a nice presentation about the changes from SYSmark 2001 to 2002. You can see a pretty outrageous selective bias.

http://www.vanshardware.com/reviews/...on%20FINAL.pdf

-----

This isn't related to back then....

AMD, Nvidia, VIA are simply mad that the GPU Acceleration has no impact on the test score

Mad enough to leave

CPU performance has no coincidence to this

Rumor:
(AMD Bulldozer is heavily mirrored on to the Intel Compiler thus it would perform better)