Why are reviewers so lazy? ( Not talking about TPU!!! )

[ShurikN]

Its funny how OP mentions game choices, and says not aimed at TPU, yet reviews on this site are done on BF1 instead of 5, CoD WW2 instead of BO4, Rise of TR instead of Shadow, AC Origins instead of Odyssey...

 

[W1zzard]

Its funny how OP mentions game choices, and says not aimed at TPU, yet reviews on this site are done on BF1 instead of 5, CoD WW2 instead of BO4, Rise of TR instead of Shadow, AC Origins instead of Odyssey...

Just wait for next round of rebench, should be soon. Takes almost two weeks of nonstop benching - during which I can't do other graphics card reviews

 

[phanbuey]

Might be time to recruit minions? That sounds brutal.

 

[cucker tarlson]

You do have a point with benching brand new, just released games. NVIDIA almost always has an advantage there because they tend to have a "GameReady" driver out for it before it launches (which benchmarkers often use) where AMD tends to be a month or two behind. First benchmarks are, therefore, going to have an NVIDIA bias.

lol,what silly thinking AMD drags their feet with releasing optimized drivers so day one reviews are nvidia biased cause they're ready with a driver.Really? Is that the definition of reviewers bias or is that the definition of AMD,being late with releases most of the time ?

Its funny how OP mentions game choices, and says not aimed at TPU, yet reviews on this site are done on BF1 instead of 5, CoD WW2 instead of BO4, Rise of TR instead of Shadow, AC Origins instead of Odyssey...

cause all he wants is to crap on people who don't do it the way he wants. he is a grown version of a spoilt child.
he says nothing about the fact that reviewers (including tpu) often choose to benchmark dx12,which favors AMD,whereas dx11 runs better for nvidia and most of the times beats the dx12 score of amd easily,only complains about instances where testing methodology is not specifically tailored for AMD cards. His aim is not to urge reviewers to be more "honest",as he put it,he's just mad that some of the tests are not favoring amd cards the way he wishes.

He's referring to 1080 and V64 percentage differences while urging that TPU review games at medium settings cause "10% of gamers only play at Ultra".Really ? 10% of people with V64 or 1080 play on Ultra settings ? How did you come by these numbers.

this is what gets me mad about people like him or adoredtv.Steve from HWunboxed is out with reviews every couple of days, always works his ass of to present just about the most comprehensible set of results, while those hack tech commentators don't bother to do the tests themselves but will be there yapping their mouths when they find one thing they don't like.

why are reviewers so lazy,this thread is an abomination.some while ago I had a suggestion for w1zzard to test both dx11 and dx12 in BF1 and deus ex,I never said to him or made a thread that he's lazy.I don't know what is going on in OP's head to think he can feel entitled to point out that TPU/HWunboxed are not even trying to be good reviewers,I hope he'll rethink that and gets some perspective on the matter.

 

[FordGT90Concept]

lol,what silly thinking AMD drags their feet with releasing optimized drivers so day one reviews are nvidia biased cause they're ready with a driver.Really? Is that the definition of reviewers bias or is that the definition of AMD,being late with releases most of the time ?

As stated, the reasons for the delay are outside of AMD's control. It makes no sense to benchmark a brand new game knowing AMD performance will likely improve in a matter of weeks. The same is also true of NVIDIA's poor performance on AMD-marketed games (e.g. Sniper Elite 4).

W1zzard has said that he quit including The Division because the game ended up getting regular updates that change performance numbers. The same logic should be applied to games that released within three months.

 

[the54thvoid]

With regard to the OP's points, yes, some are valid. But I've never read a review from any site that pins down all the facts. It would be futile for a site to perform such a review unless all it did was gfx reviews.
If you have any understanding of 'reviews' you know to read multiple sources to get a balanced perspective. It's good reviewers do different things, it gives the consumer valuable information.
I like the sheer spread of games reviewed on TPU, and the comparisons you can draw from that.
Other sites give different takes.

The best thing TPU has is an attentive owner that listens to the crap people throw at him.

Hell, it's time to resurrect the April Fools when he resigned because of it.

 

[cucker tarlson]

It makes no sense to benchmark a brand new game knowing AMD performance will likely improve in a matter of weeks.

The same is also true of NVIDIA's poor performance on AMD-marketed games (e.g. Sniper Elite 4).

only nvidia takes only a couple of days to update their drivers. and I see no reason not to benchamrk nvidia's gpus when they're not ready with a final driver,not to mention the idea of waiting weeks or months for a final amd driver to test a game is absurd.
and what was the deal with sniper elite 4. I'm checking the day one reviews and it seems nvidia is doing just fine.

 

[Toothless]

Lately I've been running ultra settings on many games with a 1950X and a HD7970. I'm actually quite impressed at how well the 7970 holds up compared to my 1080s. The actual difference is quite shocking, to be honest, to see how the 7970 holds its own for me. BF:V @ 2560x1600 on a 7970? Yeah, I'm doing that.

Ive been poking around at smaller cards and a 7970 might work in my itx build. Glad to know its still beefy.

 

[remixedcat]

There are some decent youtube channels that test games and settings just like you want.

Most reviewers test games at absurd settings because they have to or they get no more review samples.

But yet twitch THOTS get review samples and stuff This one got a ryzen threadripper and a high end radeon and all she does is do things that THOTs do.

 

[FordGT90Concept]

only nvidia takes only a couple of days to update their drivers. and I see no reason not to benchamrk nvidia's gpus when they're not ready with a final driver,not to mention the idea of waiting weeks or months for a final amd driver to test a game is absurd.

The intent is to compare graphics card performance, not game stability/optimization. Using any newly minted game tests the latter more than the former.

 

[cucker tarlson]

The intent is to compare graphics card performance, not game stability/optimization. Using any newly minted game tests the latter more than the former.

then the solution for you is not to read them. plenty of people want day one tests, but there is this silly thinking again.let's rafrain from doing day one tests complettely cause amd is late with drivers.
just resort to reading gpu reviews,they include games that usually are fully updated.

 

[dj-electric]

Most reviews today are lazy because they consume enough time as is and don't reward financially anyway.
Tell me how many reviews you read talk about frame-time and consistancy of GPUs along testing?
How many of em lack detail in overclocking the product?

 

[cucker tarlson]

Most reviews today are lazy because they consume enough time as is and don't reward financially anyway.
Tell me how many reviews you read talk about frame-time and consistancy of GPUs along testing?
How many of em lack detail in overclocking the product?

to measure frame time consistency you'd need to cap all cards at the same fps.
as an example, comparing frame time comsistency of Vega 64 vs gtx 2080 is pointless,one is +40% faster,it's gonna eat up a whole lot more of cpu resources,therefore resulting in higher cpu usage and more flactuating frametimes.

 

[FordGT90Concept]

then the solution for you is not to read them. plenty of people want day one tests, but there is this silly thinking again.let's rafrain from doing day one tests complettely cause amd is late with drivers.
just resort to reading gpu reviews,they include games that usually are fully updated.

Reviews like this (game released December 4 and review was published on December 4) should be followed up in a few months.

W1zzard used 12.18.1 beta drivers which released on 11/29/2018 which "has support for Just Cause 4." There has been no optimization driver for the game released by AMD yet.

Shadow of the Tomb Raider released September 14 and it got a 3% boost in 12.18.2 released December 19. Even DOOM is still getting love from AMD in 12.18.2 (5% uplift) and that's a 2016 game.

 

[micropage7]

Most reviews today are lazy because they consume enough time as is and don't reward financially anyway.
Tell me how many reviews you read talk about frame-time and consistancy of GPUs along testing?
How many of em lack detail in overclocking the product?

sometimes i see they just show what readers want to see, like games that used for testing and yes there some limitation, some reviewers may not capable enough to push the card properly coz many aspect that need to get counted

 

[FordGT90Concept]

Reviewers are usually sent golden samples so how much they overclock is usually moot compared to the average graphics card people buy.

 

[Vayra86]

How are you completely oblivious to the fact that your statement is 100% false. In some games yes, in other games HELL no.
So looking at these results, anyone would say even a 1070Ti is better for 4K gaming compared to the Vega 64. But let's ignore the fact the framerates are practicly unplayable wich makes the entire chart useless.
Now test the same game on medium on 4K and what do you know, now the game can actually be played on a budget and the Vega 64 is only loosing to the most expensive cards on the market.
Now, 4K gamers that buy this game, are they more likely to run it on Ultra, or on Medium?


View attachment 114068

View attachment 114066

And no it's not the only game, have a look at the latest tombraider:

Ultra, 1070Ti wins against Vega 56 and Vega 64 is eaqual to GTX 1080, but framerates are barely OK by todays standard.
Medium, Vega 56 wins from 1070ti, and Vega 64 beats GTX 1080 and framerates are enjoyable.

View attachment 114074
View attachment 114075

Now you can offcourse start an argument that you don't need 60fps or higher and that PC gamers want quality over framerate. Well then, then I have no counter argument becides the fact that almost every best selling monitor in Amazons Top 50 at the moment is a 1080p high hz monitor with one exeption, a 4K Freesync monitor. that's right, freesync, not Gsync.



Offcourse people play on Ultra settings. I'm not saying they don't. But do you dare to make the statement that Ultra is the most selected setting out of them all? Or do you dare to admit that low/medium is probably more likely based on the most selling hardware ( looking at amazon for example, or looking at steam hardware survey )?

Hey look, you found Medium benchmarks! Well done! So they dó exist.

HWInfo is your place to be!

 

[cucker tarlson]

Hey look, you found Medium benchmarks! Well done! So they dó exist.

HWInfo is your place to be!

pclab.pl purepc.pl computerbase.de

oh look, vega 64 gets destroyed at high preset and lower,even though it keeps up at max,I wonder why no nvidia fanboy is ranting about testing odyssey at medium settings
https://www.computerbase.de/2018-10/assassins-creed-odyssey-benchmark-test/

https://www.purepc.pl/karty_graficz...n_s_creed_odyssey_pc_grecka_tragedia?page=0,6

https://pclab.pl/art77466-5.html


the only perso that's lazy here is the OP,there's plenty of sites that test what he wants.

 

[Bones]

I think I understand what the OP is asking........ Maybe not but here's what I'm getting from it by what I've read.

They want a direct comparison of what was benched/tested one year ago vs the newest card available today under the same exact conditions to show what the supposed performance increase "Is" over the older card.
Same OS, game, drivers, settings..... As if you suddenly had the newest card available back when the first test was done to see what the real differences would be to support any claims of performance increase over the older card tested at that time.
Also asking for settings like most gamers would probrably use as an everyday average joe comparison.

This just isn't feasable for a site like TPU or any of the so-called reviewers to do.

If the OP wants to do such testing I suggest the OP just have at it - Apparently you have the need so feed it by doing what you are proposing yourself and publish your results.

TBH, no reviewer/site is going to cater to you over such because there is no benefit from doing that.

Others have already pointed out the makers largely determine what they want from these tests and unfortunately it's the makers that have a say or the reviewers no longer get these examples to test with.
The thing about testing is to see what the card is truly capable of hence the high/ultra high settings used to get whatever the card is capable of period and from a marketing standpoint it makes sense to get the numbers up as high as possible because higher numbers just looks better when you look at them. It also shows what the card CAN do if pushed, all else aside.

Yeah, I do agree it would be nice to see such done but sorry - Ain't happening the way the OP wants it, he'll have to do it himself or forget it and that's the truth of it.

 

[Vayra86]

That is not all I've read is wrong with reviewers/reviews.

- We need driver version tests
- We need every resolution and every quality setting (or at least, 2 out of 4 quality settings broadly available; medium+Ultra)
- We need to include every game OP plays, it could be BFV today, but maybe its CoD tomorrow! Oh no wait, Fortnite is popular, we need Fortnite - never mind the fact it runs on a toaster and is completely pointless to bench!
- We need to revisit this whole test sequence every time AMD releases a driver update.
- We need to revisit the whole test sequence whenever the game the OP plays gets a big update.

I'm out. This is going nowhere, and this topic is a complete mess. Maybe @JRMBelgium try it again with a new topic, and a clear OP supported by a healthy bit of research so you can actually make a point, and not touch on five things you really would like to see in reviews.

Better yet, do the work and show people what you want, instead of what you're cobbling together now which is a vague assortment of cherry picked Vega versus 1070ti/1080 performance gaps at different games, settings and points in time. Its not helping you make a case.

Reviews like this (game released December 4 and review was published on December 4) should be followed up in a few months.

W1zzard used 12.18.1 beta drivers which released on 11/29/2018 which "has support for Just Cause 4." There has been no optimization driver for the game released by AMD yet.

Shadow of the Tomb Raider released September 14 and it got a 3% boost in 12.18.2 released December 19. Even DOOM is still getting love from AMD in 12.18.2 (5% uplift) and that's a 2016 game.

Seriously man. AMD is making their driver teams cheap to run by omitting crucial performance updates or sits on them for MONTHS, its no more than fair that they're missing the boat in a review at any point in time. Its their choice for going the el-cheapo route... and more often than not, the performance gains in the driver are heavily overstated. 8-9% turns out to be more like the 3-5% you mention. And that is *across FIVE years* of drivers... I mean... 3%. Who the hell cares. I still haven't seen any examples of GPUs that are moving up or down the hierarchy because of driver updates.

Its a strange world when reviewers across the globe are supposed to revisit work they've already done because one GPU vendor chooses a lax update schedule. Sure, you could revisit them, but we're talking about a minority marketshare vendor here. The target audience for these tests is not so very big as people might think - the vast majority isn't going to go back and read what a driver update will do because they already have a GPU that they are updating, and can experience it themselves.

The key part of this whole topic is this:
I have yet to see examples (sources!) of games that were unplayable and turned playable through a driver update. Those examples are rare and 99% of the time the problem was the game and not the driver. Until that happens, I don't see how we gain anything from revisiting games every few months. And for 'medium' - sufficient sources have been provided that show Medium test results.

 

[Vya Domus]

I personally simply do not fully take for granted any of these benchmarks, which is what everyone should do.

 

[Bones]

Good point about the drivers Vario - Even an update of drivers such as what's released to support the newest card vs the older one will make a difference in results. One would have to rerun tests with the newest drivers with both card to show such a comparison and there is no guarantee the newest drivers would jive well with the older card - I mean it's supposed to but with each new arch things change, that's how it is with no absolute guarantee of anything aside from it working..... But we know how that works out sometimes too.

As for any such reviewing - I know one thing, it won't be me doing it.

 

[JRMBelgium]

I'm happy to see some people understand what I'm trying to say. Some of you think I'm asking better reviews from a selfish standpoint. Let me tell you this. I personally think BF V sucks bigtime. And I prefer BF1 and like it 10x more. That doesn't change the fact that the BF1 population is now a fraction of the BF V population, and in order to adress as much gamers as possible, the BF V benchmark makes more sence.

Some of you are defending flawed review methods out of respect for the reviewers here that work hard with limited time and budget. I understand that. But there are reviewers out there that make 10x the money Wizz is making and they do have lots of people working for them, and they still use the same flawed review methods. You have to be honest and admit that comparing two peaces of hardware, on different software and putting them in the same chart is not a good way to compare them. That's like testing two cars on a different road and then concluding that one car needs more fuel for the same distance. Testing older games when newer titles are released with a population 5-10x great also doesn't make sence. You try to say it as much as possible that I want to force my opinion on to others. But when a new title has 10x players then an old title, you are writing an article that could be 10x more relavant for those gamers.

And when a game performance article gets written, and the author knows that those numbers are no longer correct due to a game patch or driver update, it only takes 5 minutes to edit the article and mention that there have been performance improvements in the latest patch/driver and that the article no longe. represents current performance. You can defend Wizz or whatever reviewer you respect or like. But respecting someone does not mean that you have to admire them like a god or ignore what he or she does wrong or could do better.

This is my final post since people cant post arguments without getting personal with me or anyone else who tends to agree or understand what I'm saying. Asking me to start reviewing hardware knowing full well that it's impossible for a normal consumer to do that is just...

 

[Bones]

This thread just needs to be locked..... I don't see anything of any benefit happening from here guys.
Out.

 

[FordGT90Concept]

I have yet to see examples (sources!) of games that were unplayable and turned playable through a driver update. Those examples are rare and 99% of the time the problem was the game and not the driver.

Witcher 3 and tessellation come to mind.