Why are we not seeing Video Card reviews using other games?

[EastCoasthandle]

For example a few from the following:
-Dirt
-Graw2
-The Chronicles Of Riddick <-awesome
-Lost Planet
-Call of Juarez
-Test Drive

Others include:
-Tiger Woods PGA Tour 07
-FIFA Soccer 07
-NBA Live 08
-Flight Simulator X Deluxe
-Close Combat: First to Fight

Just to name a few. I am not saying that all of these be used. What I am saying is that everyone doesn't/no longer plays Fear, BF2142, Quake 4, etc. A little diversity would be nice.

[HellasVagabond]

Generally sites use games that are most known to people to conduct benchmarks. Another thing is that they may have not figured out a way to run benchmarks with some of the above games.

[quickie]

I would imagine most review sites use games that will really stress the video card and titles people are familiar with. I don't think Tiger Woods PGA Tour 07 would give many cards a run for their money

[[I.R.A]_FBi]

DiRT would ..

[mortal]

I think that they are using games that are very popular, and you can buy it everywhere.

[mandelore]

well... we are trying to get benchies etc for call of jaurez.. but no1 else wants to contribute here on tpu

[KennyT772]

Originally Posted by [I.R.A]_FBi

DiRT would ..

You cannot record a time demo or save a replay. In order to benchmark you have to create a set, reproducible path in order to get consistant framerates. Some games such as WoW or oblivion do not have this capability but you can easily reproduce a section just by running there. This is not the case with DiRT.

[sneekypeet]

Originally Posted by quickie

I would imagine most review sites use games that will really stress the video card and titles people are familiar with. I don't think Tiger Woods PGA Tour 07 would give many cards a run for their money

BTW i beg to differ...If i have my system or cards just a little too high 07 will BBSOD my machine where farcry and TOCA will continue to play at those clocks. The only other one i have that may be a bit tougher is COD2. Just my 2 pennies...nothing personal!

[EastCoasthandle]

It would be more diverse to see other titles benchmarked.

[newtekie1]

Well the problem is that if they were to add new games they would have to retest those new games with all the old cards previously reviewed so they can have comparisons, which is one of the main points of a review.

Now the problem is that a lot of the reviews are done on cards that aren't actually owned by the reviewer. They are review samples that are given to the reviewer so they can do the review, then they have to give the card back. So they can't go back and retest with the new games, so they can't have comparisons.

Usually a review site picks a set of benchmarks and stick with them to avoid this problem.

[Dippyskoodlez]

Originally Posted by EastCoasthandle

It would be more diverse to see other titles benchmarked.

Would be nice to see a more diverse set of games with a benchmark mode too.

[W1zzard]

i think none of these games except for COJ offer any kind of benchmarking mode.

playing each game on each card and then using fraps will just take way too long and wont be precise

if you have suggestions for games which can be automated i'll gladly add them

[EastCoasthandle]

Originally Posted by W1zzard

i think none of these games except for COJ offer any kind of benchmarking mode.

playing each game on each card and then using fraps will just take way too long and wont be precise

if you have suggestions for games which can be automated i'll gladly add them

True, part of the problem is that the games don't have a benchmarking tool. However, I do know of a place that has. HOCBench
is just one place that I stumbled upon that offers some sort of benchmarking tool for some games.

[Grings]

can you save replays in colin mcrae?, cause then you would have a virtual timedemo (each run of it would be exactly the same)
If i was w1zz i think i would just wait for some benchable directx 10 stuff to come out

[mandelore]

call of juarez has a dedicated (and seperate) dx10/9 benchmark program

[von kain]

Originally Posted by mandelore

well... we are trying to get benchies etc for call of jaurez.. but no1 else wants to contribute here on tpu

i know i was curious to see a dx 10 bench but no one really cares...
there is any generic dx 10 game on the stores?? because i want to see the really difference in gpus on dx10 ??








p.s. hd2900xt 320 stream processors must be something on dx10 cause on dx9 isn't

[Mussels]

i did video card reviews for a while, and actually the big killer is this - if you use new games in every review, how do you compare it to the other cards you've tested? Your reviews might be a year or two apart, and without using the same programs and the same (or similar) test rig you have no comparison to draw your own conclusions, all you have is FPS numbers with no meaning.

The other reason, is that its very time consuming - if you have a set amount of benchies that you know well, you can learn any flaws/screwups they do and you're fine

Example: Company of heroes 1.70 patch had a memory leak, that repeated runs of the benchmark caused an FPS drop each run. This made reviewers results skewed, as they usually went lowest to highest on the graphics, and the results ended up slower than they should have been. If i threw that in randomly without having tested it first, the results would be completely different to what you'd see testing it yourself, and to actual gameplay.

would it be possible, do you think, to start flight simulator x, tip your joystick on it's side (for an ever-lasting loop), then test with fraps? as long as you stopped the test on time, this wouldn't be too hard<.<

[W1zzard]

Originally Posted by EastCoasthandle

True, part of the problem is that the games don't have a benchmarking tool. However, I do know of a place that has. HOCBench
is just one place that I stumbled upon that offers some sort of benchmarking tool for some games.

uh ya .. i wrote tpubench http://www.techpowerup.com/tpubench which has a ton of plugins and works better for me than this hoc stuff

[KennyT772]

Originally Posted by Grings

can you save replays in colin mcrae?, cause then you would have a virtual timedemo (each run of it would be exactly the same)
If i was w1zz i think i would just wait for some benchable directx 10 stuff to come out

I already said you can't if you are referring to DiRT that is. Colib Mcrae 05 isn't the most stressful game anymore.

[Mussels]

Wizzards TPU bench is awesome :P and it backs up what i said about a standard set of benchmarks, every piece of hardware reviewed here can be compared to others in the same tests.

Bassmasta: Yes, if you want an FPS reading that you cant compare. You cant guarantee its the same every time around (as in, EVERY frame is the same) so you get variations. Theres no point in doing a test like that, if its random how its going to turn out because then you cant compare it to another card/system.

Originally Posted by Mussels

Wizzards TPU bench is awesome :P and it backs up what i said about a standard set of benchmarks, every piece of hardware reviewed here can be compared to others in the same tests.

Bassmasta: Yes, if you want an FPS reading that you cant compare. You cant guarantee its the same every time around (as in, EVERY frame is the same) so you get variations. Theres no point in doing a test like that, if its random how its going to turn out because then you cant compare it to another card/system.

fair enough, but as I recall, 3d mark doesn't always give the same test, either. I recall the ragdoll test having a physics and a quality test, neither of which ever turned out exactly the same.

[Urlyin]

Originally Posted by bassmasta

fair enough, but as I recall, 3d mark doesn't always give the same test, either. I recall the ragdoll test having a physics and a quality test, neither of which ever turned out exactly the same.

in most cases you run the bench several times and take the average...

[Mussels]

and which is why those tests arent part of the default runs either :P