• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.
  • The forums have been upgraded with support for dark mode. By default it will follow the setting on your system/browser. You may override it by scrolling to the end of the page and clicking the gears icon.

techPowerUp 2011 CPU benchmark suite

Omega

Omega

New Member
Joined
Sep 9, 2009
Messages
140 (0.02/day)
Location
Sibenik - Croatia
Hi again :)
First of all, I'd like to thank all of you for support regarding CPU review section, which we have been running almost for a year now. As many of you may have noticed, reviews have evolved greatly since their first iterations, and in the process revealed many bugs and limitations in list of benchmarks currently used.

In addition to some apps being outdated and unable to show full potential of new processors, others have consistency issues with results and had to be deleted from benchmark setlist (audio encoding).

In few months time, we will have a small processor evolution on our hands with the coming of AMD's Bulldozer and Intel Sandy Bridge architectures, and techPowerUp! needs to be ready for them. Currently, a new benchmark list is under construction that will need to hold ground in the next year or so, most of the processors will be retested, some models will go to history archives...

What I need from you, readers and forum members, is opinions, ideas, criticism. We've done quite a bit reviews in the past year and we're planing to keep it up in the future, or rather do it even better than before. As I said few months back in similar thread, these reviews are made for you and because of you.
So feel free to voice your opinion, what works and what doesn't in our reviews. What would you like to see more and what less. And most important, what benchmarks would you like to see processors struggle with, what games would you like to see us use in testing, and so on.

Bellow is the list of known components that will be used for testing. Not much has changed from what we used until now, and the goal is to keep it as simple and cost effective it can be, without any danger of limiting processor potential. I know many of you would like to see a stronger graphic card in game testing, but since we are testing in resolutions up to 1680x1050 with medium/high graphic details without AA/AF, graphic card should rarely bottleneck the CPU performance. Remember, the goal here is not to score as much FPS you can, rather to have results comparable between different processor models where it's possible to see what impact on graphic performance faster/slower processor has.


Motherboard:
AMD: MSI 890FXA-GD70
Intel LGA1366: unknown
Intel LGA1156: ASUS P7P55D PRO
Cooler: Scythe Katana III
Memory: 2 x 2048 MB Mushkin PC-16000 DDR3 @ 1333 MHz 7-7-7
Graphic card: VTX ATI Radeon HD 5850
Harddisk: ~ 250GB classic hard disk
Power Supply: Enermax Liberty 620 W
Software: Windows 7 Ultimate 64-bit


As for mentioned benchmark setlist, bellow you can find the current list of benchmarks and applications used. Some of them will remain there in the future, like synthetic benchmarks (newer versions), rendering and so on.
The goal is to have balanced setlist, that will stress out every aspect of modern day processor, and that includes synthetic benchmarks for show of theoretical potential, performance with multimedia files, applications that mimic professional use like rendering, apps to test archiving and general use performance, and of course gaming performance.

Again, any opinion is more than welcome and every suggestion will be taken into consideration, and will be tested for practicality. If found useful, it will be added to future benchmarks. In a couple of days I'll post new benchmark setlist from my point of view, and we can build something concrete from there.

The areas I need help the most would definitely be choice of games (maximum of 5 games, all must have in game benchmarks or 3rd party benchmark tools), and audio encoding tools (multithreading support is a must).

Thanks!

Benchmark Suite 2009/2010​
  • Everest Ultimate v5.30 - Read/Write/Copy/Latency
  • PC Mark Vantage v1.0.1 64-bit
  • 3D Mark Vantage v1.0.1 Performance preset (CPU and Total score)
  • wPrime v2.0 1024M
  • SuperPI 1M
  • bonk encoder/dbpoweramp - deleted
  • I'm Too Audio Encoder v2.1.78 - WMA encode with LAME MP3 128 kbs
  • x264 benchmark HD V3.0
  • Handbrake v0.9.4 DVD rip to .mkv (Standard High preset)
  • Xilisoft Video Converter Ultimate v5.1.26 - .mov 1080p encode with H.264/MPEG4 AVC .mp4 preset
  • DivX Converter 7.2 - AVI encode with Home Theater preset
  • Photoshop CS4 Retouch Artists Speed Test
  • Cinebench R10 64-bit
  • Blender 2.49
  • POV Ray 3.7b 64-bit
  • 7zip v4.65 64-bit - 32 MB mutithreading
  • WinRAR Benchmark v3.91 64-bit
  • WinRAR real life compression ~ 1 GB folder
  • Crysis Warhead v1.2 64-bit
  • Modern Warfare 2
  • Warhammer II v1.5
  • DiRT 2
  • GTR Race ON!
  • Resident Evil 5 benchmark
  • Prime95 for maximum heat and power consumption
 
check out civilization V. it has a benchmark mode and high cpu usage
 
also windows experience index cpu might be useful. just run "winsat cpu" from the command line, easy to automate
 
Lame 3.99 might be worth considering for MP3 encoding software tests, both 32 and 64bit versions are available and it is Multithreded. There were some limitations apparently with earlier versions upto 3.98 but 3.99 seems to be popular.
 
Maybe include a couple of Browser benchmarks? Nevermind which browser performs better, but they all benefit from better CPUs, right? :)

now that I think about this, it would probably only be important for low-end CPU tests... but it's still an idea ;)
 
Maybe rather some kind of complete office benchmark that includes browser performance?
 
You could possibly add GTA IV as one of the games as that scales very well with different processors and would show some relative differences in them.
 
I'd rather go with Mafia II for that kind type of game.
 
Omega said:
Maybe rather some kind of complete office benchmark that includes browser performance?
Click to expand...
That would be great, :)
You know of any office benchmarks that include browser performance? :D I don't
 
Wyverex said:
I don't
Click to expand...

Not the response I've hoped for :D
Haven't got a clue, but I'll google for it as soon I get some free time... in the meantime, maybe somebody will save me the trouble :)
 
Could I suggest ARMA or World in Conflict? Both are very cpu-intensive...
 
Suggestion noted.
But I'm not very fond of those titles. Did some testing with World in Conflict, results can be very buggy when changing hardware, and as for ARMA, isn't the whole game flamed due to large number of bugs?
 
Yeah, they are far from perfect, but two of the most cpu-limited titles out there at this point. Anything else, you've already covered. ;)
 
might be interesting to see something like a Sony Vegas encode, or something. But even then, it only takes about 15 minutes to render 1 hour of video @ 720p on Q6600...

OF course, I'm always interested in multi-gpu performance impacts...
 
I have a few comments about the test setup.

First, could you get a SSD instead of a HDD? Is there a reason why you haven't chosen this route? (cost, size?) Anandtech said somewhere that benching with a SSD produces more consistent results.

Also, the video encoding benchmarks all seem to overlap, that is they do the same thing. The first two both use x264 to encode, the third still uses some kind of H.264 encoder and I'm pretty sure DivX nowadays uses H.264 as well. If it doesn't, then I think Mpeg-4 encoding is a little dated anyway. All those tests could easily be shrunk into one, maybe two tests. Personally, I like the x264 benchmark because I can do it myself and compare results.
 
angelkiller said:
First, could you get a SSD instead of a HDD?
Click to expand...

Ha! Great question, I was waiting for that one actually :)
Few days ago I had a chat with W1zzard about that one, and it's true that with SSD you get more consistent results, checked it myself. I do have one Crucial 128 GB SATA III SSD available - but I don't know for how long, that would be the first problem.

The second one, as W1zzard pointed out, is the SSD's performance degradation over time which could start to affect the results in say one year-year and a half, if our benchmark list would hold out that long.

Yes, you could just replace it after a while, but those things are expensive as hell. On the other hand, full partition image backup on classical hard drive with each hardware change has proven itself so far. It's an open question, one that will be closed at the very end when everything else is decided.

As for video encoding, I could not agree more. Any suggestions besides x264 benchmark ?
 
Dirt 2 Because of it being DX11, Metro 2033 It would be nice to see how the new ATI cards do with some much tessellation. Just Cause 2 a game that uses Allot of resources for being DX10, Crysis, F1 2010 will have DX11 attributes with an update., BaTMan, Civilization V Because DX11.

There are some I think would be nice :)
 
WCG has a CPU Benchmark tool and results would be helpful to WCG users. :toast:

WCG = World Community Grid

This could help our team grow here at TPU! :toast:
 
Omega said:
Ha! Great question, I was waiting for that one actually :)
Few days ago I had a chat with W1zzard about that one, and it's true that with SSD you get more consistent results, checked it myself. I do have one Crucial 128 GB SATA III SSD available - but I don't know for how long, that would be the first problem.

The second one, as W1zzard pointed out, is the SSD's performance degradation over time which could start to affect the results in say one year-year and a half, if our benchmark list would hold out that long.

Yes, you could just replace it after a while, but those things are expensive as hell. On the other hand, full partition image backup on classical hard drive with each hardware change has proven itself so far. It's an open question, one that will be closed at the very end when everything else is decided.

As for video encoding, I could not agree more. Any suggestions besides x264 benchmark ?
Click to expand...
I'm sure we could get you a SSD. Ask for $1 donations. Seriously. A lot of people will chip in $1 and I'm sure bunch more would contribute more.

Performance degradation is a valid concern. But you have to weigh that versus the variability and possible bottleneck of a HDD. I think a degraded SSD would still produce better (more reliable) results than any hard drive. I also think that the HDD's variability would affect the results more than a SSD's performance degradation would. (I think I said the same thing twice)

You also could combat the performance degradation. Instead of testing on a new drive, you could fill the drive with random data or whatever then format and bench. This way, you purposely put the drive in a degraded state before each bench and this state can be regenerated whenever necessary. Hopefully that makes sense.

And as for video encoding benchmarks, I unfortunately do not know of any more. H.264 seems to be the most widely used codec. Everything else seems to be inferior or not used very often. (Divx, Xvid, Mpeg-2, WMV??)

Edit: Here's a link to Anandtech showing how they simulate a 'used drive' or make the drive perform in its worst case scenario, which makes the test repeatable.
 
Last edited:
Maybe a VMWare Player benchmark. Just something quick, like installing Win Vista or 7 in a virtual machine, then use this to test boot time with the different processor in a virtual envirnment. Maybe even run a few other benchmarks in the virtual enviroment.

And once you set it up the first time, the virtual machine can just be copied from one machine to the other.

Virtual environments are getting more popular, so I feel it would be nice to have at least a little testing done on how the different processors handle virtual computing.

Also, I'd like to see 7-zip real world compression performance also, since in my experience it has been better at taking advantage of multi-threaded processors than WinRaR.

Omega said:
I think none of those you mentioned has built in benchmark modes?
Click to expand...

Batman Arkham Asylum does, and you can use the CPU for hardware accelerated PhysX to really test the CPU.;)
 
Last edited:
You must log in or register to reply here.
Back
Top