Why are reviewers so lazy? ( Not talking about TPU!!! )

[Vayra86]

Shocking! Saved you from a BSOD then, maybe. My point was that multi-monitor isn't very "isolated." NVIDIA went full damage control mode.

I will concede that Nvidia has not been pushing out high quality driver releases ever since Windows 10. But the two are not entirely unrelated either, and the way Microsoft has implemented their updates is not ideal either when you put it next to Nvidia's release approach, which is preferably through GFE and their own version history instead of what MS uses.

But... the cool thing is that Nvidia pushes out a new driver so often, you're never stuck with a big issue for longer than a few weeks.

Regardless, we're horribly offtopic because none of this has anything to do with reviewing and performance changes over time.

I'm not sure the horse is dead yet, though. People might as well keep beating it.

IKR.

Still waiting on those examples I've asked for six pages now... unplayable games becoming playable on any GPU at any point in time, because of driver updates.

 

[cadaveca]

I dont recall ever getting a golden chip/card/board from intel/amd/nvidia or board/card partners. Though I do believe this happens at times, it's not as often as some make it out to be. Most results fit within a nice average and arent outside the norm in my experience reviewing over the last decade. So while we may not "know" just looking around at reviews one can get an idea that it is not typical...otherwise those preaching about this would be constantly crying about it and we dont see that often.

I most definitely recall getting cherry-picked samples.

Different time, I asked Intel for three 4960X... one high leakage, one low leakage, one in the middle. I got exactly that, within a couple of days.

But it is very obvious when these parts are given, and most of us know when it happens, but not all.

I agree with your general stance though.

I'm not sure the horse is dead yet, though. People might as well keep beating it.

You know what would make this a non issue, just in general? if in a review, it was possible to have every performance metric you could think of, and when you viewed a review, it automatically adjusted the results to match your own personal configuration, so the results shown would be the exact same results you would get? When I did reviews, I really strived to make that possible... anyone can easily replicate every benchmark and result in the reviews I wrote... if you have the same hardware. things like turbo clocks and such on both CPUs and GPUs make this very complicated.

Reviews cover "relative performance". Until they can show people "real performance", then this horse is gonna get beat over and over and over....

IKR.

Still waiting on those examples I've asked for six pages now... unplayable games becoming playable on any GPU at any point in time, because of driver updates.

GRAW, BF3, BF4, BF1, BF:V, and a whole slew of other titles that are not played with launch-day drivers can have all sort of issues. With you making a claim like htis.. I wonder, do you even play games?

Drivers are so important to performance and stability in games... People are still waiting for DX12 driver updates for BF:V, for example, because as it is now, it IS largely unplayable. Pretty much any title ever release that featured multiple DX rendering paths has had huge impacts to performance from driver updates.

 

[Vya Domus]

Still waiting on those examples I've asked for six pages now... unplayable games becoming playable on any GPU at any point in time, because of driver updates.

Can happen, years ago when I had a 5770 and Rage came out it was actually unplayable as in single digit framerates and horrendous graphical artifacts. It was only fixed by a driver update later on.

 

[Vayra86]

GRAW, BF3, BF4, BF1, BF:V, and a whole slew of other titles that are not played with launch-day drivers can have all sort of issues. With you making a claim like htis.. I wonder, do you even play games?

Drivers are so important to performance and stability in games... People are still waiting for DX12 driver updates for BF:V, for example, because as it is now, it IS largely unplayable. Pretty much any title ever release that featured multiple DX rendering paths has had huge impacts to performance from driver updates.

Yes, it happens, but are those the drivers / games we see in review results? When a combination has a major issue, it is dismissed for reviews, or revisited later on in a game performance article. Even very recently with the DXR implementation for BFV. And when DX12 is horrendous, a different API is used or they're both included.

I think reviewers are pretty close to the subject matter and pretty accurate in the subjects they write about and what the audience is asking for.

An example; I also frequent Tweakers.net, and they had a Witcher 3 reviewed with Hairworks on. Myself and a number of others commented on that, and within a few days the review was corrected with Hairworks off results.

Reviews cover "relative performance". Until they can show people "real performance", then this horse is gonna get beat over and over and over....

Thank you! That is exactly the point. People want something simplified for them and are simply unwilling to interpret the data. They want it fed to them in their exact preferred shape and size.

Its not the reviewers being lazy... its the reader. Even the OP commented and linked two TPU reviews and completely missed the fact they were done on the same test rig, even when W1zz explicitly replied to the same guy that this is how he does his testing. The proof is right here in this topic...

Here it is
https://www.techpowerup.com/forums/...azy-not-talking-about-tpu.251199/post-3970595

 

[FordGT90Concept]

I will concede that Nvidia has not been pushing out high quality driver releases ever since Windows 10. But the two are not entirely unrelated either, and the way Microsoft has implemented their updates is not ideal either when you put it next to Nvidia's release approach, which is preferably through GFE and their own version history instead of what MS uses.

But... the cool thing is that Nvidia pushes out a new driver so often, you're never stuck with a big issue for longer than a few weeks.

Microsoft really has nothing to do with it other than setting clearly defined goals (Redstone 1/2/3/4/5). AMD can meet those goals while NVIDIA can't?

 

[Vayra86]

Microsoft really has nothing to do with it other than setting clearly defined goals (Redstone 1/2/3/4/5). AMD can meet those goals while NVIDIA can't?

Good point, I can't contest that one.

 

[cadaveca]

Thank you! That is exactly the point. People want something simplified for them and are simply unwilling to interpret the data. They want it fed to them in their exact preferred shape and size.

Its not the reviewers being lazy... its the reader. Even the OP commented and linked two TPU reviews and completely missed the fact they were done on the same test rig, even when W1zz explicitly replied to the same guy that this is how he does his testing. The proof is right here in this topic...

Here it is
https://www.techpowerup.com/forums/...azy-not-talking-about-tpu.251199/post-3970595

Sure. But I think you missed something here... the OP wasn't specifically trying to call out TPU for this problem, and in fact, excused TPU from the start. I think the OP was trying to convey a general idea about dis-satisfaction with how reviews are presented, but could only use specific example to explain how they felt, and guys like you took it specifically rather than generally. That's not the OP's fault, IMHO. It's something I've seen happen here on these pages for years now.

I don't think it's readers being lazy. It is that understanding the information as its related... that is not easily done. You can tell me one thing does 50, and the next 75, and I understand 75 is a bigger number, but WHY is it important? How do you QUANTIFY what that "performance" actually offers? Reviews do not do this very well at all.

I see it as the job of the reviewer to meet the needs of the reader. There are no lazy readers, ever. Simply put, the content provided doesn't meet their interest well enough. But as it goes "you can't please all of the people, all of the time"... but as I said earlier, that doesn't mean no one should ever try.

 

[rtwjunkie]

and in fact, excused TPU from the start. I

No he didn’t. You have a reading comprehension problem. Last paragraph in the OP. Go read it again. It’s funny you are the only one who missed that.

 

[Aquinus]

Sure. But I think you missed something here... the OP wasn't specifically trying to call out TPU for this problem, and in fact, excused TPU from the start. I think the OP was trying to convey a general idea about dis-satisfaction with how reviews are presented, but could only use specific example to explain how they felt, and guys like you took it specifically rather than generally. That's not the OP's fault, IMHO. It's something I've seen happen here on these pages for years now.

No, he called out TPU for not doing reviews at medium quality.

And this feedback is aimed at Techpowerup directly, stop benchmarking at the highest quality settings and start benchmarking at medium. The amount of gamers that play on Ultra only represents 10% at most. There are settings like shadow/lighting or post processing that almost every gamer turns down for the big performance impact or the disadvantage it gives in visibility. So why benchmark settings that 90% of gamers will not use.

Then it turned into driver versions, then different types of hardware. I'm all for reasonable ideas, but good ideas also have to be weighed with common sense because it's not like resources are infinite.

 

[TheoneandonlyMrK]

Actually it's fine with whatever criteria that they use as long as it's valid
And i don't agree test in medium setting why coz you can't see its power, if it is capable in high setting with high AA and AF it should be perform well in medium

I think the opposite, more is better, low , middle and highest set tests imho would apeal to a wider audience at the level they game at, it adds a lot of tests but I have often thought tpu has an extensive game list matbe compartmentalize them with three games used with varying game quality settings , and a different three with different res settings or more extensive resolution testing.

The site's made to pull people in, ideally more new people, displaying a bias towards high end gamers must intimidate visitors surely.

 

[cadaveca]

No he didn’t. You have a reading comprehension problem. Last paragraph in the OP. Go read it again. It’s funny you are the only one who missed that.

I didn't miss it. I filtered it with the title in mind. That's not a reading comprehension issue on my part. It's a lack of understanding on yours, IMHO.

To that point specifically, I previously mentioned here that I am gaming on a 7970. Do you think Ultra means anything to me? Sure, it does when I plop in my 1080, but I can see the OP's frustration easily. Few people can afford the hardware required to game on Ultra settings in many games to begin with.

Then it turned into driver versions, then different types of hardware. I'm all for reasonable ideas, but good ideas also have to be weighed with common sense because it's not like resources are infinite.

Naturally. But again, how is one to relate an idea without a specific example? You bet resources are limited... but the limit to those resources is one defined and imposed by W1zz's choices.

And to bolster that idea, i'll just point you in the direction of TPU's recent Instagram and Patreon ventures... a lack of resources isn't the problem there.

Now, that said, I do agree with you when commenting about W1zz's reviews. There's nothing wrong, and they relate relative performance, always. No big deal. But that doesn't mean I can't understand how someone may want something different, and I'm not going to deride them for asking for it... I've got you guys to do that for me.

 

[rtwjunkie]

I didn't miss it. I filtered it with the title in mind.

No, you came late to the party. If you filtered his very open jab at TPU in the last paragraph based on the title, then I have to tell you, you are completely in left field.

The OP specifically changed the thread title to cover up for the fact that he was catching a lot of grief because he specifically aimed his sight on TPU in that paragraph, and then presumed to call out @W1zzard to his face because the standards were not to his liking

For whatever reason you are choosing to be the OP’s champion, to the point you ignore very explicit statements made by the OP. If you don’t believe me by the way, about the thread title change, ask a super mod. I’m pretty sure they can tell you the thread history.

I’m out of here. It’s pointless when such a determined champion refuses to give up on his liege, even when said person is doing exactly what the champion says they aren’t.

 

[Vayra86]

Sure. But I think you missed something here... the OP wasn't specifically trying to call out TPU for this problem, and in fact, excused TPU from the start. I think the OP was trying to convey a general idea about dis-satisfaction with how reviews are presented, but could only use specific example to explain how they felt, and guys like you took it specifically rather than generally.

I'll agree I could have been more subtle to open up with, but what has also occurred on page one is that people pointed out this will create impossible amounts of work, and on top of that, the guy was factually wrong with a number of examples while also questioning the legitimacy of TPU reviews. The topic title was changed, 'not talking about TPU' was added later. But the overall tone of voice was: you're deluding us with your reviews, and the examples used were lots of Vega performance improvements on a recent driver.

Not exactly a good opening post if you want to make a point and have a good discussion.

I don't think it's readers being lazy. It is that understanding the information as its related... that is not easily done. You can tell me one thing does 50, and the next 75, and I understand 75 is a bigger number, but WHY is it important? How do you QUANTIFY what that "performance" actually offers? Reviews do not do this very well at all.

+1. Reviews dó have a job to do when it comes to 'guiding' the readers. Performance is misunderstood all over this forum, too, on both ends of the spectrum. And reviews do lack certain niches in gaming that are heavy on performance in a specific way, for example leaning heavily on CPU - late game strategy is a good example. And about quantifying the differences in FPS - correct - and in that sense I feel TPU has a job to do in for example showing min. FPS more consistently alongside averages. And perhaps in some text to specify what is considered and felt 'playable' and what didn't. But re-testing and rebenching a dozen games in several quality settings continuously as updates are rolled out? Meh. Not sensible, and above all, really not worth the effort in a broad sense. It also doesn't help in 'quantifying' anything, its just more numbers.

So, this I can agree with. Up to a point, though. That point is when readers blatantly do NOT read what is being pointed out to them explicitly. W1zzard specifically replied to him with points about accuracy and it was taken in the stride and largely ignored, only to reiterate false statements such as a lack of medium setting testing, while several sources already existed showing those tests. And a slew of other carefully selected facts that were ignored because they didn't sit well with the OP. Thát is also not a good basis for discussion.

 

[cadaveca]

No, you came late to the party. If you filtered his very open jab at TPU in the last paragraph based on the title, then I have to tell you, you are completely in left field.

The OP specifically changed the thread title to cover up for the fact that he was catching a lot of grief because he specifically aimed his sight on TPU in that paragraph, and then presumed to call out @W1zzard to his face because the standards were not to his liking

For whatever reason you are choosing to be the OP’s champion, to the point you ignore very explicit statements made by the OP. If you don’t believe me by the way, about the thread title change, ask a super mod. I’m pretty sure they can tell you the thread history.

I’m out of here. It’s pointless when such a determined champion refuses to give up on his liege, even when said person is doing exactly what the champion says they aren’t.

Nah, it's just as I said. I am filtering. I'm not saying you are wrong, at all. I'm merely saying the OP has a point about a lack of coverage by all sites in general. I don't care what anyone thinks about TPU, so nothing anyone says in that context has me react, is all. I don't care if people don't like TPU... so I see nothing to defend. Perhaps you forget I posted many reviews while NOT doing what commenters asked for, all the time.

So now that the drama has passed, rather than focus on it, as I feel you are, I was rather hoping to discuss the actual topic, and not have that discussion dismissed because you don't agree with one part of it. It's not being the OP's "champion". LOL. That's funny.


But i will say this: Everyone that has ever made any money has done so by providing what someone wanted, but could not have. So that has me wonder about the OP's request, drama removed.

 

[Solaris17]

But let's ignore the fact the framerates are practicly unplayable

Damn, 49 FPS in a game makes it unplayable. PC gaming has changed under my nose.

 

[LightningJR]

I didn't read this whole thread but I will say I like the idea of a Medium setting tested, I know it requires extra work. If I were to give an opinion I would say that at 4K resolution any card that can't pull off 60FPS ave should be added to the 4k med benchmark as well.

I wouldn't be mad if all cards that couldn't get 30fps ave fps was moved to a medium preset benchmark with all other cards that can't stay above 30fps while having 1 card overlap (both med and ultra settings)

All of this though adds work. What I like and what I can get are different things, but I would like it none the less lol.



I will saay though that the 4k Med benchmark makes the most sense to me, so few cards can pump out high FPS at 4k so for people who want to know if they can get decent FPS at medium might be beneficial.