Why are reviewers so lazy? ( Not talking about TPU!!! )

[Vya Domus]

Some early CPUs are much worse than a typical consumer one due to rushed production or immature node.

I am baffled as to how you can so vividly claim anything at all about this. You don't know what Nvidia/AMD/Intels/partners/retailers send and how that compares to the rest of 99.99% silicon out there. Stop it, you have no foundation for your claims.

 

[rtwjunkie]

Who here doesn't use multiple monitors?

I don’t, for one. You’re as bad as the OP, making generalizations about what people use or don’t use.

 

[FordGT90Concept]

how many times has your card bsod ?

Never:
Radeon
Radeon 64
Sapphire 9800 Pro
PowerColor X800 XL
Sapphire HD 5870
PowerColor 290X
PowerColor 390
Sapphire RX 590

The HD 5870 had an unstable factory memory overclock. That resulted in full screen artifacts rather than BSODs. I eventually put reference VBIOS on it and it has been stable ever since. Sapphire is to blame for that one, not AMD.

Sometimes a manufacturer a asks a big chain of local stores to hand a unit for a review, and there's absolutely 0 control over the quality of sample sent.

If the hardware ever hit the retail ecosystem, it isn't going to be an intentional golden sample.

I don’t, for one. You’re as bad as the OP, making generalizations about what people use or don’t use.

Shocking! Saved you from a BSOD then, maybe. My point was that multi-monitor isn't very "isolated." NVIDIA went full damage control mode.

 

[dj-electric]

Interesting, that's a lot of review hardware. So where are the reviews you've done? Who is "media"?

HWzone.co.il based in IL.

I am baffled as to how you can so vividly claim anything at all about this. You don't know what Nvidia/AMD/Intels sends and how that compares to the rest of 99.99% silicon out there. Stop it, you have no foundation for your claims.

Have tested so many CPUs and GPUs the only thing i can suggest is taking my word for it, just like i'm suggesting taking anybody else's word for it

I know how every Nvidia, AIB, intel and AMD is sent to me. I have all the shipping information regarding goods in invoices. I compere results i get with a few people who do this prior to releases sometimes just to check if everything is in line, and results typically make sense. i compere results with a few friends that own stores locally to also see the deviation.

I can know if a sample is typical, gold, or just doodoo like it has been many times before.
I see people claiming that all samples sent for reviews are golden and im just laughing knowing that sometimes it is very far from truth.

 

[Vayra86]

Back to the topic at hand: benchmarkers choose "Ultra" it enables all of this AMD breaking GameWorks trash. Suddenly AMD graphics cards look awful in performance reviews when it is just NVIDIA's unoptimized code rearing its ugly head again.

Nah, every reviewer that picked Hairworks in their review got burned hard for it and either redid the test within the same article OR turned it off to begin with.

Hairworks meant a major performance hit on EVERY card. Only Maxwell cards were hit a bit less, but even those had abysmal performance. You completely missed that line...

Please, keep it factual.

 

[cucker tarlson]

here we are talking about testing and holding people up to impossible standards,yet some members learned nothing from the discussion.

Back to the topic at hand: benchmarkers choose "Ultra" it enables all of this AMD breaking GameWorks trash.

that is very incorrect factually.

 

[FordGT90Concept]

Nah, every reviewer that picked Hairworks in their review got burned hard for it and either redid the test within the same article OR turned it off to begin with.

Hairworks meant a major performance hit on EVERY card. Only Maxwell cards were hit a bit less, but even those had abysmal performance. You completely missed that line...

Please, keep it factual.

There was a whole lot of media frenzy about it back when Witcher 3 launched. Example:
AMD Radeon graphics run Witcher 3 just fine, unless Nvidia HairWorks is enabled--and that's OK

 

[Vya Domus]

I have all the shipping information regarding goods in invoices.

Do they include silicon quality in there ?

Have tested so many CPUs and GPUs

You don't understand that this does not matter at all. "So many" is still dozens of orders of magnitude less that what is being shipped out there.

I see people claiming that all samples sent for reviews are golden

People suspect that is the case but that can't be proved by anyone. Just like how you cannot prove that early CPUs are of much worse quality than consumer ones.

 

[dj-electric]

Hey, i ain't gonna try and convince you further that review samples are mostly completely RNG. You have a sense of judgement and should believe what you think is truth.

When a motherboard + CPU combo comes in, they include a sheet of tests with that CPU, and the required voltage for stability.
Last one that came in was Z390 Pro with a 9700K and the damn thing needs 1.32V for 5Ghz. Not exactly optimal

GPUs came in in complete randomness from local importers or from PR agencies. Haven't seen an above average RTX card so far, for example.

Just like how you cannot prove that early CPUs are of much worse quality than consumer ones.

I have not said that. I said that sometimes early CPUs actually have bad OC capabilities due to some reasons.

 

[Vya Domus]

I am also going to say that, with all due respect, I don't know how careful Intel/Nvidia/AMD is going to be with what they send through who to your site compared to people like TPU/Linus/
JayzTwoCents etc. They definitely do care about what sort of coverage they get and from who.

 

[Aquinus]

GPUs came in in complete randomness from local importers or from PR agencies. Haven't seen an above average RTX card so far, for example.

It might make a difference when you're getting hardware directly from the manufacturer and not through an intermediary. I'm pretty sure that a lot of hardware W1zz tests comes from the source.

 

[micropage7]

It's gonna get hot.
This conversation got wider

 

[FordGT90Concept]

It's also generational. Adored TV made a very, very strong case that i7-8700K was golden sampled (most reviewers got 99th percentile CPUs in terms of overclocking). Why did Intel do it? The pressure from AMD Ryzen was real. They had vested interest in overselling their chips. Only manufacturers know how often they golden sample products to reviewers. Journalists, as far as I know, only caught Intel doing it once in a statistically significant way. That doesn't mean there aren't other cases.

 

[Vya Domus]

It might make a difference when you're getting hardware directly from the manufacturer and not through an intermediary. I'm pretty sure that a lot of hardware W1zz tests comes from the source.

And as far as I know the FE RTX cards come exclusively from Nvidia.

 

[Aquinus]

And as far as I know the FE RTX cards come exclusively from Nvidia.

I would expect CPUs to come from AMD and Intel as well a lot of the time.

 

[FordGT90Concept]

And as far as I know the FE RTX cards come exclusively from Nvidia.

Which has always been a huge red flag for me. They can golden sample cards to the media. AMD could too with their Frontier Edition.

 

[dj-electric]

compared to people like TPU/Linus/

You should ask W1zz and dave, who have tested many many CPUs over their years working at TPU regarding CPU bin. You'll get the same answers. Don't take my word for it. Take theirs Also, read the friendly reviews. Those CPU overclocks past maybe 10 reviews were far, far from impressive.

 

[Vya Domus]

You should ask W1zz and dave

It seems to me you are unwilling to understand what is being told to you. I doesn't matter what they are going to say because there is no way for them to know if the product they got is representative of everyone else's. This isn't about what reviewers do, it's about what the manufacturers do. Which is clearly beyond the scope of everyone else.

It's absolutely hilarious to see how you try and convince us that manufactures have no control whatsoever over what they send to who.

 

[dj-electric]

It's absolutely hilarious to see how you try and convince us that manufactures have no control whatsoever over what they send to who.

They have very tight time frames to work with. They can barely test the samples for stability most of the times before they send it.
If they have control, than it is quite shitty. Where can i refund or swap the past maybe 10 CPUs i got? Most of them OC'd absolutely horrible compered to retail units. Intel knew well a head of time, yet failed to provide an above avg unit.

 

[EarthDog]

Most reviewers test games at absurd settings because they have to or they get no more review samples.

lol, no.

It might make a difference when you're getting hardware directly from the manufacturer and not through an intermediary. I'm pretty sure that a lot of hardware W1zz tests comes from the source.

I dont recall ever getting a golden chip/card/board from intel/amd/nvidia or board/card partners. Though I do believe this happens at times, it's not as often as some make it out to be. Most results fit within a nice average and arent outside the norm in my experience reviewing over the last decade. So while we may not "know" just looking around at reviews one can get an idea that it is not typical...otherwise those preaching about this would be constantly crying about it and we dont see that often.

 

[micropage7]

I am also going to say that, with all due respect, I don't know how careful Intel/Nvidia/AMD is going to be with what they send through who to your site compared to people like TPU/Linus/
JayzTwoCents etc. They definitely do care about what sort of coverage they get and from who.

yeah, but i've stopped watching Linus since he shout and yelling alot, i miss when he unboxed stuff

 

[HD64G]

I am for not using ultra settings for GPU comparisons but I disagree with medium also as it creates CPU bottlenecks that reduce the performance diff between GPUs. Very high mixed with some high settings are the best imho. Ultra usually use some features-gimmiks that increase the load much more than increasing graphic quality (i.e. gameworks from nVidia).

As for @W1zzard not having tested the new AMD drivers for months, I think he will do so with the 1st stable driver of 2019 sometime close to February. And indeed after testing them myself, AMD has increased performance for quite a few games (not only the latest) in their drivers.

 

[Vayra86]

lol,what silly thinking AMD drags their feet with releasing optimized drivers so day one reviews are nvidia biased cause they're ready with a driver.Really? Is that the definition of reviewers bias or is that the definition of AMD,being late with releases most of the time ?


cause all he wants is to crap on people who don't do it the way he wants. he is a grown version of a spoilt child.
he says nothing about the fact that reviewers (including tpu) often choose to benchmark dx12,which favors AMD,whereas dx11 runs better for nvidia and most of the times beats the dx12 score of amd easily,only complains about instances where testing methodology is not specifically tailored for AMD cards. His aim is not to urge reviewers to be more "honest",as he put it,he's just mad that some of the tests are not favoring amd cards the way he wishes.

He's referring to 1080 and V64 percentage differences while urging that TPU review games at medium settings cause "10% of gamers only play at Ultra".Really ? 10% of people with V64 or 1080 play on Ultra settings ? How did you come by these numbers.

this is what gets me mad about people like him or adoredtv.Steve from HWunboxed is out with reviews every couple of days, always works his ass of to present just about the most comprehensible set of results, while those hack tech commentators don't bother to do the tests themselves but will be there yapping their mouths when they find one thing they don't like.

why are reviewers so lazy,this thread is an abomination.some while ago I had a suggestion for w1zzard to test both dx11 and dx12 in BF1 and deus ex,I never said to him or made a thread that he's lazy.I don't know what is going on in OP's head to think he can feel entitled to point out that TPU/HWunboxed are not even trying to be good reviewers,I hope he'll rethink that and gets some perspective on the matter.

Thank you for this summary because it is spot on. We are feeding the troll here and probably should just stop...

I will say it again: there are no trends here that change a thing about a buyers decision when it comes to overall GPU performance, gpu hierarchy or it changing over time. If it did, that would be a real reason to radically change the approach to reviews. So far Im still missing sources that prove they do consistently... only in some rare exceptions that offset one another in both camps.

TL DR its a non issue, concluded already on page 1 yet here we are...

 

[Aquinus]

TL DR its a non issue, concluded already on page 1 yet here we are...

I'm not sure the horse is dead yet, though. People might as well keep beating it.

 

[londiste]

The problem is time and effort it takes to benchmark. Even disregarding problems with hardware change limitations - as is the case with Battlefield V - my recommendation is to try and run a wide enough set of benchmarks with even remotely acceptable methodology on 2 GPUs. Just to see how long it takes and get some idea what this entails. For better idea, run it on some common resolutions like 1080p, 1440p and 2160p.

You will need to run each benchmark at least 3 times to basically get a minimum consistency, more if you can. Many benchmarks are not easily scripted.
There are things to be said on topic of built-in benchmarks vs gameplay - and finding a repeatable, consistent and representative gameplay benchmarking is far from easy.
If you intend to benchmark frametimes, doing this the right way requires some hardware for recording and analyzing these. Requirements on space and time are not trivial.

Now, try multiply this with all the awesome ideas - running this with different CPUs (2, 4, 6, 8, 12, 16 cores would be awesome for determining scaling), different settings (low/medium/high plus various optimized settings), different RAM amounts (8/16/32 would be nice), different drivers (all the drivers since launch would be overkill but awesome).

Offtopic on golden samples:

AdoredTV used SiliconLottery to confirm Intel sends golden sample CPUs to reviewers:

This is going offtopic but his analysis methods are very VERY flawed there. I'll check if I can find my spreadsheet with the table of 8700K reviews and OC results. IIRC the actual results from sites were (marginally) worse than SiliconLottery's - which is not unexpected given that there are some ES CPUs sent to reviewers.