Benchmark project - PCI-E videocards since 2004

[havli]

This is my 5th VGA benchmarking project. The preparation was ongoing for a long time and now it is finally the time to launch it.

But first a brief descriprion of the previous four projects. The first one started in 2007 and took about 1.5 months to finish. So in fact a very small project... but it was just a start.

Project #1
Time frame: 13th Ocrober 2007 - 22nd November 2007 (1.5 months total)
VGAs of years: 1996 - 2001 (14 pieces total)
Testing platform: Celeron 600, Athlon XP 2000+
Games / applications used: 9
Amount of settings per card and game: unknown
Link: dead

Project #2
Time frame: 23rd June 2009 - 5th March 2011 (22 months total)
VGAs of years: 1998 - 2001 (65 pieces total)
Testing platform: Athlon XP @ 2.16 GHz
Games / applications used: 15
Amount of settings per card and game: up to 15 (640x480x16, 800x600x16/32, 1024x768x16/32, 1600x1200x16/32, 800x600x16/32 2xAA, 800x600x16/32 4xAA, 1024x768x16/32 2xAA, 1024x768x16/32 4xAA)
Values measured: ~ 5900
Link: http://hw-museum.cz/article/2/benchmark-vga-1998---2001--2011-edition-/1

Project #3
Time frame: 7th December 2010 - 15th March 2016 (66 months total, project stopped, ~ 80% finished)
VGAs of years: 2004 - 2008 (50 pieces total)
Testing platform: Core i5 2500k @ 4.5 GHz
Games / applications used: 20 (+ many of them tested at two levels of details)
Amount of settings per card and game: up to 8 (1024x768, 1280x1024, 1600x1200, 1920x1200 - all of them at noAA, 16xAF / 4xAA, 16xAF )
Values measured: > 10000
Link: http://hw-museum.cz/article/3/benchmark ... edition-/1 Partial results, last update was sometimes in 2012. Anything measured after that never made it to the public and never will.

Project #4
Time frame: 20th October 2013 - present (59 months, project still active, ~ 80% finished)
VGAs of years: 2000 - 2004 (58 pieces planned)
Testing platform: Pentium E5700 @ 3.9 GHz
Games / applications used: 19 (+ many of them tested at two or three levels of details)
Amount of settings per card and game: up to 12 (800x600, 1024x768, 1280x1024, 1600x1200 - all of them at noAA, noAF / 2xAA, 4xAF / 4xAA, 8xAF)
Values measured: ~10000
Link: Not online yet, will be published as an article on my website (no ETA though ). Until then there will be series of GPU Duel videos on Youtube.

----------------------------------------------------------------------------------------

Enough of history, now it is time for the latest project. I have been testing PC hardware over 10 years and during that time I gained a lot of experience in this field. Also my capabilities of preparing the right SW tools for proper benchmarking are now better than ever. Not to mention the ability to buy the right tools and equipment which wasn't the case before.
This means the Project #5 wil be the best I can do... and I dare to say it won't be far from the best GPU tests you can find on the whole Internet. Also I realized "less sometimes means more" and truly monstrous projects like #3 here may prove too difficult and time consuming to finish in reasonable timeframe. In this case it all took too long, the methodology became obsolete. So this time there will be slightly slimmer, reasonable time demanding methodology - more quality, less quantity.

The base stone is no doubt the test system. This time it is rather old, but proven by the years of service. Also it matches all my requirements - support of 2-way SLI/CF 16+16 and 3-way SLI/CF 16+8+8. So we are looking at Sandy Bridge-E X79 platform - Asus P9X79 Deluxe and Xeon E5-1650 overclocked to 4.8 GHz. I also considered using something more up-to-date. But there are not many more options in HEDT it seems. Haswell-E wouldn't be that much faster (better IPC, lower clock) and still costs a lot when I want the 40-lanes variants. Gaming performacne of X299 and X399 also isn't all that great and not very cost-effective anyway.


The final test platform:

Xeon E5-1650 @ 4.8 GHz at 1.4V, HT disabled (SilentiumPC Navis PRO 240 AIO)
Asus P9X79 Deluxe
4x4 GB DDR3 1600 CL9-9-9-24-1T
Toshiba 3 TB HDD (7200 rpm) - storage for games
Kingston HyperX Savage 120 GB - system
Corsair RM1000i
Fractal Design Define S (4x 140mm, 2x 120mm on AIO)
Windows 7 64 SP1
Acer XF270HU (2560x1440)

List of video cards to benchmark:

Considering the problems with getting the right system for the test, it seems this one will have to last not only for the first part of this test but also for planned part two or even the part three.

Part 1
Part one is starting with GeForce 6 / Radeon X800 and up to G80 and R600. Also I think some AMD APUs will fit this part nicely - Llano (FM1) and Kabini (AM1).

Part 2
Part 2 is still in a distance but the table of video cards to take part is pretty clear for me. It is still work in progress but more or less what you see here will be tested. I think Radeon HD 4770 shoud be also in there and of course some APUs as well. I think Llano will be good match for this part and also Kaveri.

Part 3
This one is still far far away Still, it will continue with DX11 HW. So most likely Starting with HD 5770 / GTS 450 and up to at least R9 290X / GTX 780 Ti. Perhaps I'll extend it up to Fury X and GTX 980 Ti.
No table here, as it is still too early for that.


Methodology

List of games and SW for part one (2004 - 2007)
3DMark 03, 05, 06
Doom 3
Far Cry
Far Fry 2
Half-Life 2 EP2
NFS Most Wanted
Bioshock
Call of Duty 4, 5
Crysis
ETQW
Serious Sam 2
FEAR
Mirror's Edge

Resolution and settings for part one (2004 - 2007)
1024x768 16xHQAF, noAA
1024x768 16xHQAF, 4xMSAA
1280x1024 16xHQAF, noAA
1280x1024 16xHQAF, 4xMSAA
1280x1024 16xHQAF, 8xMSAA
1280x1024 16xHQAF, 4xMSAA + TRAA
1920x1080 16xHQAF, noAA
1920x1080 16xHQAF, 4xMSAA
1920x1080 16xHQAF, 8xMSAA
1920x1080 16xHQAF, 4xMSAA + TRAA
2560x1440 16xHQAF, noAA
2560x1440 16xHQAF, noAA

Of course I won't test all combinations. For instance with some games AA or TRAA isn't working. And second rule (which saves a ton of time and also my eyes ) - anything that get less than 15 fps avg is out and will get a nice zero instead. Noone would play games at less than 15 fps anyway.


Measured properties

First I must admit, these two are out of my reach. Proper equipment for it is simply too expensive and/or difficult to get.

Frametimes measuring using FCAT.
Measuring power consumption of video card isolated from the rest of the system - just like here on TPU or Tom's HW, etc.


The rest shouldn't be so hard, sot now the better part of the list:

Approximately 80% of all values beasured using Fraps.
Those 80% will contain frametime analysis of course.
Measuring power consumption at wall socket.
Measuring power consumption inside the PSU via SW utility in the OS.
Measuring everything MSI Afterburner can do - GPU load, VRAM, CPU load, RAM, temperature, etc.
Log all the above mentioned values to files and afterwards create charts.
Noise measuring idle / load.
Temperature measuring - 10 minutes Crysis, then 10 minutes in desktop.
Recording some benchmarking videos using second PC and capture card (zero influence on the measured HW) at 1080p / 60 fps. This should be good enough to actually see microstuttering and other smoothness issues.

So all in all this methodology should match the best reviews you can see on the net.


Processing of the measured results and presentation

Although this project is means to be less time consuming than the previous ones, it will take time to get everything done and then write an article. Because of that I decided to publish GPU Duels on Youtube on the fly.
How such duel looks? Well, I took an inspiration in GPUReport's Incredible Benchmarks series:

Webwalker (the author of GPUReport) did really great job here. I made my own application that creates video similar to this one from the Fraps and Afterburner logs mentioned earlier. It took a lot of work to get it done but now it is almost ready and you can expect the first Duel very soon.

When the part one is complete, it will be available on my website http://hw-museum.cz Of course Excel-made charts are now somewhat obsolete, so I have my own tools. The result should be easily readable and it will hold all the interesting extra informations... not just fps.

I will update this thread with news about the testing progress. And also post links to the GPU Duel videos. You can expect the first one in few days.

 

[havli]

Ok, here we go - the first GPU Duel is ready.

Radeon X1900 XTX vs GeForce 7950 GX2

Now when the SW tool to create this is finally complete, I can have some rest. It may not look that difficult, but it took well over 100 man-hours of development to get this done.

 

[R00kie]

holly shit, subbed

 

[havli]

Round 2

Radeon X1900 XTX vs GeForce 8800 GTS 640

 

[Solaris17]

holy shit my dude. subbed.

 

[havli]

This time more balanced fight:

GeForce 8800 GTS 640 vs Radeon HD 2900 XT

 

[havli]

New GPU Duel is online:

GeForce 8800 Ultra OC vs 2 × Radeon HD 2900 XT

 

[phill]

Outstanding

 

[agent_x007]

@havli Doesn't fully enabled G80 have 64 TMUs ?

Each block of 16 SPs shares 4 texture address units, 8 texture filter units, and an L1 cache.

Les 64 unités de filtrage portent donc leurs fruits. (eng. "The 64 filtering units are therefore bearing fruit.")

Source : LINK + LINK.

 

[havli]

I'm certainly no expert at understanding GPU architectures. But There are opinions that G80 is considered to have only 32 TMU... as it have 32 texture address units.
https://www.techpowerup.com/forums/threads/confused-by-number-of-tmus-nv-g80.213248/

 

[agent_x007]

My problem with 32 TMUs, is actual Texture Fillrate that you can measure :

Max. texture fillrate = 36,9GTex/s (with 64 TMUs), and ~18,5GTex/s with 32 TMUs.
Vantage measured : 17,1GTex/s, so...
17,1/18,5 x 100% = 92,4% efficiency.
Valid : LINK.

Here are G92 numbers :

Max. texture fillrate = 47,2GTex/s (with 64 TMUs), and ~23,6GTex/s with 32 TMUs.
Vantage measured : 21,91GTex/s...
21,91/23,6 x 100% = ~92,8% efficiency.
Valid (not mine, but feature tests are in-line) : LINK.

So, does this mean G92 is also 32 TMU design ?
If that is true, I think you should stick to one number for all Tesla class NV cards.

 

[havli]

Well, the 3dmark measuring can be inaccurate.... Anyway, it is just a number - some say it is 32, others say 64... you can flip the coin to choose which one you want to use.

For example Fermi architecture has also different amount of texture filtering and texture addressing units. And for Fermi always the lower ones are used as TMU count.
https://en.wikipedia.org/wiki/GeForce_400_series

Each SM in the GF100 contains 4 texture filtering units for every texture address unit. The complete GF100 die contains 64 texture address units and 256 texture filtering units[6] Each SM in the GF104/106/108 architecture contains 8 texture filtering units for every texture address unit. The complete GF104 die contains 64 texture address units and 512 texture filtering units, the complete GF106 die contains 32 texture address units and 256 texture filtering units and the complete GF108 die contains 16 texture address units and 128 texture filtering units

 

[agent_x007]

Interesting...
Let's stick to lower number then
Thank you.

 

[havli]

Looking at my 3dmark 06 G80 fillrate numbers it also seems too suggest there are more than 32 TMU. Well, I just like the lower numbers and always used them. Luckily as far as I know, only G80 has questionable amount of TMUs.

For example various 3DM06 fillrate single/multi figures of video cards measured so far:

Radeon X850 XT PE = 3616 / 8617 (16 ROP / 16 TMU @ 540 MHz)
Radeon X1900 XTX = 4997 / 10324 (16 ROP / 16 TMU @ 650 MHz)
Radeon X1950 XTX = 6151 / 10324 (16 ROP / 16 TMU @ 650 MHz)
Radeon HD 2900 GT = 3976 / 7186 (12 ROP / 12 TMU @ 600 MHz)
Radeon HD 2900 XT = 8057 / 11796 (16 ROP / 16 TMU @ 743 MHz)

GeForce 7900 GS = 5491 / 9901 (16 ROP / 20 TMU @ 500 MHz)
GeForce 7950 GT = 5706 / 12605 (16 ROP / 24 TMU @ 550 MHz)
GeForce 8600 GTS = 2729 / 7726 (8 ROP / 16 TMU @ 675 MHz)
GeForce 8800 GTS 640 = 5147 / 12149 (20 ROP / 24(48) TMU @ 513 MHz)
Quadro FX 4600 = 6054 / 11925 (24 ROP / 24(48) TMU @ 500 MHz)
GeForce 8800 Ultra = 7783 / 20328 (24 ROP / 32(64) TMU @ 650 MHz)

 

[agent_x007]

No, it doesn't.
20328 is within max. theoretical for 32 TMUs at 650MHz (20800).
Just like X850 XT PE for example.

 

[Vayra86]

This is amazing, Thank you for the effort and for sharing!

 

[Dinnercore]

This is incredible work, I was kinda looking for something like this and thought about making it myself but did not have the patience and time.

Thank you so much for all the effort!

 

[havli]

Thank you all for support

 

[havli]

Another video - Radeon X1950 Pro vs GeForce 8600 GTS

 

[Vayra86]

Another video - Radeon X1950 Pro vs GeForce 8600 GTS

Those frametime differences, wow. And now I understand why AMD users said their video output was better, look at the color difference. The AMD side is much more vivid and saturated.

Its really hit or miss in terms of frametimes in both camps depending on the game. You don't see that as much these days, overall much has improved.

 

[phill]

Remember testing Far Cry 2 with a 4870 X2.. And playing Serious Sam 2 with a QX9650 and a 8800 GT !! Ah, the memories Probably still got the benchmark results as well somewhere lol

 

[havli]

Radeon X1900 XTX vs Radeon X1800 XT... feel the power of extra pixel shaders

 

[framebuffer]

Fantastic work!
Also the "Benchmark VGA 1998 - 2001 (2011 edition)" is very interesting and brings up a lot of memories

 

[havli]

GeForce 8800 GTX SLI vs Radeon HD 2900 XT CF

 

[basco]

thank you very much for this !
looking forward to the equal cards with different memory size comparison.
if ya need a card like 3850x2 or 5870 2gb or newer or others pm me