QA Consultants Determines AMD's Most Stable Graphics Drivers in the Industry

[wiak]

Tell AMD to use opengl on pcsx2. It can get unstable quickly.

you do know pcsx2 is not developed by amd?

 

[efikkan]

A WHQL'd driver should never fail, that's kind of the point.

Well, I tought you knew more about software than this. WHQL is not much more than a "rubber stamp", and both AMD and Nvidia have managed to ship fatally flawed drivers with WHQL certifications.

GTX 1060 runs the same driver code as GTX 1050 and GTX 1080 Ti, so when GTX 1060 keeps failing the tests and the others don't, it gives a good indication of a bad card rather than a bad driver. And it's really easy to eliminate, just use a larger sample size than 1. If there is a driver issue, then there should be a pattern. Also, when using a total of 6 different cards from the two vendors, any random piece of data would impact the conclusion, and the results could just as easily have tilted the other way.

 

[natr0n]

you do know pcsx2 is not developed by amd?

The article is about AMD's drivers stability.

I stating that I can prove them to get unstable rather quickly with pcsx2 using opengl.(In which AMD's drivers are rather shit for.)

Thank You.

 

[JRMBelgium]

Go go go AMD!

 

[bug]

Well, I tought you knew more about software than this. WHQL is not much more than a "rubber stamp", and both AMD and Nvidia have managed to ship fatally flawed drivers with WHQL certifications.

GTX 1060 runs the same driver code as GTX 1050 and GTX 1080 Ti, so when GTX 1060 keeps failing the tests and the others don't, it gives a good indication of a bad card rather than a bad driver. And it's really easy to eliminate, just use a larger sample size than 1. If there is a driver issue, then there should be a pattern. Also, when using a total of 6 different cards from the two vendors, any random piece of data would impact the conclusion, and the results could just as easily have tilted the other way.

I will add to that that my 1060 has been rock solid. Lab tests meet real life

 

[efikkan]

I will add to that that my 1060 has been rock solid. Lab tests meet real life

Sure I've put three GTX 1060s into systems myself, including one of my own.

But I want to point out that unstable hardware is way more common than most people realize. I'm confident that if every buyer did similar stress tests of their new hardware, a good portion of them would identify stability issues. Also, I've seen hardware experience a lot of stability issues during development.

 

[Midland Dog]

Tell AMD to use opengl on pcsx2. It can get unstable quickly.

i blame pcsx for that, i had a bad experience with burnout 3 on pcsx2 with OGL and im an nvidia user, just use d3d for gaming, ogl can perform but it takes dev time to make it do so

 

[R-T-B]

And for any research someone interested pays always.

Not always a biased party though. There is a clear difference between a true academic grade study, and this.

 

[Xzibit]

Not always a biased party though. There is a clear difference between a true academic grade study, and this.

Unless companies decide to get along and fund a independent firm to conduct test, these back and forth between companies are the closest thing

Not like reviewers are going to test them or fund these test. They rely on freebies/samples to continue to operate.

 

[R-T-B]

Unless companies decide to get along and fund a independent firm to conduct test

It's still not in the same league as legit research. This raises eyebrows all over no matter how you cut it, and that's just from the get go.

I say that fully believing AMD drivers HAVE improved, mind.

EDIT: Just understood what you are saying. Yes, I do acknowledge that it doesn't happen often people or companies fund tests like this out of the goodness of their heart.

 

[RejZoR]

Just because someone pays the tests, that doesn't mean the tests are biased. Such tests are quite common in security segment, particularly antiviruses. There is however a catch that party paying for the test can decide not to publish the results if they are unfavorable to them. Which can still provide an internal audit for their services/products so they improve upon them and try again. If they are favorable, they usually go ahead and publish them. It's of testing body's best interest to be impartial if they want to be taken seriously for the future as testing organization. Paying them to do the testing is simply that, paying for the service of doing the tests, not paying to be shown as favorable.

As for everyone having doubts, just look at the evolution of AMD's control panel. Back in Radeon 8500 days it looked horrible. Then CCC came and while it was heavy, it was a massive leap forward. The modern interface they have now makes NVIDIA with it's "2004" interface look like a sad joke. And not only that, the control panel is super responsive, applying settings is instant and it's just nice to use and comes with OC tools out of the box.

What does NVIDIA have? Prehistoric looking interface that's all cluttered, bugged and horrendous to use, it's flashing and resetting as you apply settings, just pathetic and sad. It's just an absolute mess that I wasn't a big fan of back in GeForce 6000 series times and same applies since I got back with NVIDIA with GTX 980.

As for driver itself, I've had many problems on GTX 980 and GTX 1080Ti which required months to get fixed. But they fixed them eventually (finally). About the same as experience with AMD up till HD7900 that I had last. So, on that front, they are about the same in my book. But that control panel difference...

 

[bug]

Just because someone pays the tests, that doesn't mean the tests are biased.

I don't think the beef with these tests is that they're biased. The beef the low sample used and the opacity about what exactly was marked as failed by each test.

 

[RejZoR]

Well, QA Consultants seem to be working for 20 years in this field. I'd say they are doing something right and I don't think they'd go and ruin their reputation for AMD only. But then again, I don't know their work specifically.

 

[bug]

Well, QA Consultants seem to be working for 20 years in this field. I'd say they are doing something right and I don't think they'd go and ruin their reputation for AMD only. But then again, I don't know their work specifically.

I will give them the benefit of the doubt. But the engineer in me just doesn't take things at face value

 

[efikkan]

I don't think the beef with these tests is that they're biased. The beef the low sample used and the opacity about what exactly was marked as failed by each test.

Exactly.
Completely unbiased tests are rare, but increasing the sample size makes it harder to manipulate the results, and even better if the third party source the cards themselves.

Well, QA Consultants seem to be working for 20 years in this field. I'd say they are doing something right and I don't think they'd go and ruin their reputation for AMD only. But then again, I don't know their work specifically.

Sure, but consultants are also experts at making reports and documentation full of BS. I have no issue with AMD paying for the test, but the quality of the test which is buried in the report is just not good enough to make any sensible conclusion.

 

[HD64G]

Not always a biased party though. There is a clear difference between a true academic grade study, and this.

Many academic studies are paid by individuals or enterprises and sometimes have been proven to be paid just to get a pre-specified result out of the "research" in order to promote plans that help those who paid for it to win something politically or financially. So, until a research gives away strong signs of it being nonobjective we shouldn't doubt so strongly against its findings imho. Or else we should not accept any research findings until we manage to find it true ourselfs somehow.

 

[R-T-B]

Many academic studies are paid by individuals or enterprises

I mean, usually not directly. Or it wouldn't be an academic study, but a sponsored one.

Meh, there are other issues with this anyways, as others have pointed out.

 

[Xzibit]

I mean, usually not directly. Or it wouldn't be an academic study, but a sponsored one.

Meh, there are other issues with this anyways, as others have pointed out.

Strangely those complaints could be said about the MS HLK process (sample size). The main complaint i see it wasn't wide or varying. Well that's not what they (QA Consultants) were testing. Seams SOME complaining wanted the test to be out of the scope of its intent.

 

[R-T-B]

Seams SOME complaining wanted the test to be out of the scope of its intent.

It's intent was a PR splash for a claim requiring wide and varying evidence. Let's not be light about it, "MOST STABLE GRAPHICS DRIVERS" is in the headline. I mean really? What else is the intent? There are so many factors to accurately make a statement like that (as pointed out, AMD has some of the worst and most unstable OpenGL drivers around, bet that wasn't tested) and obviously it's only intent was PR.

That isn't unusual mind. But take it for what it is, and nothing more.

 

[Xzibit]

It's intent was a PR splash for a claim requiring wide and varying evidence. Let's not be light about it, "MOST STABLE GRAPHICS DRIVERS" is in the headline. I mean really? What else is the intent? There are so many factors to accurately make a statement like that (as pointed out, AMD has some of the worst and most unstable OpenGL drivers around, bet that wasn't tested) and obviously it's only intent was PR.

That isn't unusual mind. But take it for what it is, and nothing more.

Its not wrong. The headline is an excerpt of the conclusion

In our testing, both AMD gaming and workstation GPUs performed better than the comparable
NVIDIA products. Based on our testing of the aforementioned 12 GPUs, we believe that AMD has
the most stable graphics driver in the industry

 

[FordGT90Concept]

WHQL testing is really the standard in testing graphics driver stability, at least on Windows.

AMD likely did the test to see where they were at in terms of their driver stability. They had no idea what the result would be. The fact it was in their favor is the only reason why they published it. What's to say NVIDIA didn't pay the same organization to do similar testing of their own and the results were not in their favor so they kept it under wraps?

AMD has Vanguard. NVIDIA does not. AMD has been iterating on one architecture for many years. NVIDIA has had a major architecture change relatively recently. AMD had a stable Windows 10 driver available long before NVIDIA did. NVIDIA has been in tech news a lot lately for major driver issues, AMD only did once (that I recall) and it was fixed in a few weeks. These results shouldn't surprise any one.

 

[arbiter]

In our testing, both AMD gaming and workstation GPUs performed better than the comparable
NVIDIA products. Based on our testing of the aforementioned 12 GPUs, we believe that AMD has
the most stable graphics driver in the industry

That last part is main reason this test isn't worth using as TP. I don't see how 12 gpu's is a large enough test size for them to claim 1 side is most stable drivers. That isn't even getting in to issue of provided cards vs 3rd party sourced if they were sourced from say retail chain which is unknown or if AMD provided the nvidia cards as well. If this company had any decent level of creditibility they wouldn't claim they are most stable based on 12 cards tested.

 

[Xzibit]

That last part is main reason this test isn't worth using as TP. I don't see how 12 gpu's is a large enough test size for them to claim 1 side is most stable drivers. That isn't even getting in to issue of provided cards vs 3rd party sourced if they were sourced from say retail chain which is unknown or if AMD provided the nvidia cards as well. If this company had any decent level of creditibility they wouldn't claim they are most stable based on 12 cards tested.

WHQL testing requires less. i see this QA audit more of a follow up to the HLK CRASH testing. Like out in the wild, Outside the control of the respected companies QA labs which AFAIK are only required to have a 2 system verification log (some one correct me if i'm wrong on that one) for submission.

So if people complain about this test and not the wHLK cert/sign process it makes you wonder what they are really mad at.

 

[R-T-B]

Its not wrong. The headline is an excerpt of the conclusion

And as mentioned, I don't agree they have enough data to make that claim anything more than a PR stunt.

 

[FordGT90Concept]

Microsoft developed a test to qualify drivers. AMD and NVIDIA both use that test to verify drivers. QA Consultants did with that test on these cards what they would often get subjected to over a whole year. It's no different than beating the living hell out of pre-production test cars: develop tests for specific areas of concern and keep repeating it until you start seeing failures (e.g. suspension, automatic car washers, drivetrain through 100,000s of miles, etc.).

The fact it only took 12 days to see major failures on both sides is concerning. I can assure you they're not popping Champaign bottles at RTG over this: the high failure rates of Radeon Pro drivers are concerning. Coincidentally, there aren't many Vanguard people that have them.

If you're so convinced these drivers flawless, you're free to run the test as they did 72 times over.