• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Video of the Day: 23 GeForce GTX 295 Video Cards Installed in a Single Server Rig

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.77/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
correction...folding power
 

Atlas Folder

New Member
Joined
Apr 7, 2009
Messages
4 (0.00/day)
Location
Tulsa, Oklahoma
I|m sure that this guy has had a lot of fun building 6 machines into a 19 inch rack. But 6x 4 GTXs could be improved using a PCIe expansion bay. Since this is a folding farm and not GPU SLI, he doesnt need x16 slots, so an extension bay would be quite practical. He could use x4 for the cards, and easily drive 8 or 16 cards from ONE PC.
That is not correct. The nVidia driver only supports 8 GPUs. I have spent a lot of time researching this issue and at the current time it's not possible to use PCIe extenders or motherboards that have more than 4 PCIe slots for this reason. It is also not possible to run VMs on a machine with PCIe backplanes because there is no VM software yet that allows virtualization of the GPUs for CUDA. Xen is working on it though.

You can see a discussion here: http://foldingforum.org/viewtopic.php?f=38&t=9340&start=0

and here: http://foldingforum.org/viewtopic.php?f=38&t=9011

Jason
 
Joined
Jan 11, 2009
Messages
9,064 (2.09/day)
Location
Montreal, Canada
System Name Homelabs, #2 in construction
Processor i7 3930k @ 3.6 GHz | Ryzen 1700
Motherboard AsRock X79 Fatal1ty pro | AsRock x370
Cooling Dark Rock Pro 3 | Wraith Cooler
Memory 32GB (4x8gb) DDR3 2133Mhz | can't afford DDR4 yet lol
Video Card(s) GTX 970 incoming | RX 560 2gb
Storage Bunch of SSDs (HDDs coming eventually)
Display(s) Korean 1440p | Dell refurb 19"
Case BeQuiet! SilentBase 800 | some ATX case
Audio Device(s) AKG KXXX
Power Supply EVGA P2 750W | none yet

mtosev

New Member
Joined
Mar 21, 2005
Messages
1,462 (0.25/day)
Location
Maribor, Slovenia
Processor Core i7 930
Motherboard Asus P6X58D Premium
Memory 6gb SuperTalent WB200UX6G8
Video Card(s) xfx radeon hd 5970 black edition
Storage Intel X25-M 80
Display(s) Dell UltraSharp 2209wa
Case Haf 932
Audio Device(s) Integreted sound card
Software w7

sol.fides

New Member
Joined
Nov 21, 2005
Messages
65 (0.01/day)
Location
Merrit BC
System Name blimp
Processor E8600
Motherboard Asus P5K Deluxe
Cooling Scythe Infinity w/ Dual SFF21F fans
Memory 4gb ddr2-800 crucial balistix
Video Card(s) EVGA 8800GT (705*950)
Storage intel g2 80gb, x2 WD-640ls RAID0, WD-1002EX
Display(s) 2x Dell 2007WFP 20"
Case CM Stacker 830 w/ 4x140mm & 3x120mm
Audio Device(s) Soundblaster X-fi Fatal1ty FPS, Klipsch 2.1 & logitech 330
Power Supply Corsair HX1000
Software F@H 24/7
Benchmark Scores Lappy: g51vx
The earth simulator had 41 teraflops of supercomputing power. it was #1 in the world starting in 2002 till 2004. It cost about 72 million dollars. congratulations on building an equivalent for under $30,000.
 
Joined
Sep 17, 2008
Messages
567 (0.13/day)
Location
Germany
@ Atlas Folder

On daily basis, how much time do you spend controlling all your Clients? At the moment I have only 6 Clients and sometimes I get more than one down while I'm at work.
 
Joined
Jan 11, 2009
Messages
9,064 (2.09/day)
Location
Montreal, Canada
System Name Homelabs, #2 in construction
Processor i7 3930k @ 3.6 GHz | Ryzen 1700
Motherboard AsRock X79 Fatal1ty pro | AsRock x370
Cooling Dark Rock Pro 3 | Wraith Cooler
Memory 32GB (4x8gb) DDR3 2133Mhz | can't afford DDR4 yet lol
Video Card(s) GTX 970 incoming | RX 560 2gb
Storage Bunch of SSDs (HDDs coming eventually)
Display(s) Korean 1440p | Dell refurb 19"
Case BeQuiet! SilentBase 800 | some ATX case
Audio Device(s) AKG KXXX
Power Supply EVGA P2 750W | none yet
I believe he isn't a TPU member... unfortunately
 

Atlas Folder

New Member
Joined
Apr 7, 2009
Messages
4 (0.00/day)
Location
Tulsa, Oklahoma
ghost101 said:
What ppd do you get? Also whats the power usage like?
My 24 hour average is ~267,000 PPD. You can see that here.

The power consumption of a single shelf is ~1,125 Watts. I have 5 shelves and more misc equipment. Power consumption is probably around 7,000 Watts continuous. I haven't added it all up lately.

sol.fides said:
The earth simulator had 41 teraflops of supercomputing power. it was #1 in the world starting in 2002 till 2004. It cost about 72 million dollars. congratulations on building an equivalent for under $30,000.
Well thanks, but there is an important difference. True supercomputers use 64-bit floating point math (double-precision), Folding@Home makes do with 32-bit floating point math (single-precision). So yes, building supercomputers is getting cheaper fast, but there's still no free (cheap) lunch. If you need double-precision you still must pony up the cash.

mmaakk said:
On daily basis, how much time do you spend controlling all your Clients?
Not much... I can check for dead EUE clients anywhere from my iPhone by going here, it's updated every 60 seconds. If I need to make a correction all of the machines are controllable through LogMeIn and I have LogMeIn Ignition on my iPhone so I can log into the servers from my iPhone and tweak things as necessary.

I made a post a couple of weeks ago about how I control and implement all of the clients. I think it's kinda slick even if I do say so myself, but necessity is the mother of invention. Controlling all the clients individually got old very, very fast. I had to think of a better way. You can see my post about that here.

Thanks for the interest guys,

Jason
 
Joined
Aug 30, 2006
Messages
6,513 (1.25/day)
System Name ICE-QUAD // ICE-CRUNCH
Processor Q6600 // 2x Xeon 5472
Memory 2GB DDR // 8GB FB-DIMM
Video Card(s) HD3850-AGP // FireGL 3400
Display(s) 2 x Samsung 204Ts = 3200x1200
Audio Device(s) Audigy 2
Software Windows Server 2003 R2 as a Workstation now migrated to W10 with regrets.
Thanks for coming to TPU and sharing your experience Jason. Great build(s), nice rack. I can see that perhaps the multiple PC route is easier than the extension bay option.

Did I understand you are driver-limited. Therefore max 8 GPUs per PC, so you would need a minimum of 3 independent PCs to run your 24 cards? Or does the x2 thing knock that down to just 4 cards (8 GPU dies)?

PS. Write to nVidia, show them your stuff, get them to fix the drivers, and sponsor your rack.
 
Joined
Apr 7, 2008
Messages
632 (0.14/day)
Location
Australia
System Name _Speedforce_ (Successor to Strike-X, 4LI3NBR33D-H, Core-iH7 & Nemesis-H)
Processor Intel Core i9 7980XE (Lapped) @ 5.2Ghz With XSPC Raystorm (Lapped)
Motherboard Asus Rampage VI Extreme (XSPC Watercooled) - Custom Heatsinks (Lapped)
Cooling XSPC Custom Water Cooling + Custom Air Cooling (From Delta 220's TFB1212GHE to Spal 30101504&5)
Memory 8x 8Gb G.Skill Trident Z RGB 4266MHz @ 4667Mhz (2x F4-4266C17Q-32GTZR)
Video Card(s) 3x Asus GTX1080 Ti (Lapped) With Customised EK Waterblock (Lapped) + Custom heatsinks (Lapped)
Storage 1x Samsung 970 EVO 2TB - 2280 (Hyper M.2 x16 Card), 7x Samsung 860 Pro 4Tb
Display(s) 6x Asus ROG Swift PG348Q
Case Aerocool Strike X (Modified)
Audio Device(s) Creative Sound BlasterX AE-5 & Aurvana XFi Headphones
Power Supply 2x Corsair AX1500i With Custom Sheilding, Custom Switching Unit. Braided Cables.
Mouse Razer Copperhead + R.A.T 9
Keyboard Ideazon Zboard + Optimus Maximus. Logitech G13.
Software w10 Pro x64.
Benchmark Scores pppft, gotta see it to believe it. . .
The Atlas Folder, very nice folding setup.
I wonder what it will actually be crunching.
 

DaMulta

My stars went supernova
Joined
Aug 3, 2006
Messages
16,167 (3.09/day)
Location
Oklahoma T-Town
System Name Work in progress
Processor AMD 955---4Ghz
Motherboard MSi GD70
Cooling OcZ Phase/water
Memory Crucial2GB kit (1GBx2), Ballistix 240-pin DIMM, DDR3 PC3-16000
Video Card(s) CrossfireX 2 X HD 4890 1GB OCed to 1000Mhz
Storage SSD 64GB
Display(s) Envision 24'' 1920x1200
Case Using the desk ATM
Audio Device(s) Sucky onboard for now :(
Power Supply 1000W TruePower Quattro
He was short a few cards on the bottom and it really kills the look of the system:p



Just kidding, and that's one hell of a folding rig you have!
 

HammerON

The Watchful Moderator
Staff member
Joined
Mar 2, 2009
Messages
8,064 (1.88/day)
Location
Up North
System Name Cruncher #1 / Cruncher #2
Processor 3900X / 3900X
Motherboard ASUS ROG Crosshair VIII Hero / ASUS ROG Strix B550-F
Cooling EK Quantum Momentum Monoblock / Optimus Water Block
Memory G. Skill Trident Z Neo 32 GB 3600 / G.Skill Trident Z 32 GB 3200
Video Card(s) MSI 2070 Super Gaming X / ATI 300
Storage Samsung 960 Pro 512 GB + Mushkin Reactor 1 TB / OCZ Vertex4 238 GB
Display(s) Dell 32" Curved Gaming Monitor (S3220DGF)
Case TT Core V51 / TT Suppressor F31
Audio Device(s) On-board
Power Supply Corsair AX1200 / EVGA 650
Mouse Roccat Kone Pure
Keyboard Corsair K70
Software Win 10 Pro
Benchmark Scores Always changing~
Welcome to TPU Atlas :)
It is neat to see all that awsome hardware (especially the GPUs) working hard to find a cure :toast:
For those that want to learn more about his cause, check out the link to his website at bottom of his sig.
 
Joined
Apr 21, 2008
Messages
5,248 (1.14/day)
Location
IRAQ-Baghdad
System Name MASTER
Processor Core i7 3930k run at 4.4ghz
Motherboard Asus Rampage IV extreme
Cooling Corsair H100i
Memory 4x4G kingston hyperx beast 2400mhz
Video Card(s) 2X EVGA GTX680
Storage 2X Crusial M4 256g raid0, 1TbWD g, 2x500 WD B
Display(s) Samsung 27' 1080P LED 3D monitior 2ms
Case CoolerMaster Chosmos II
Audio Device(s) Creative sound blaster X-FI Titanum champion,Creative speakers 7.1 T7900
Power Supply Corsair 1200i, Logitch G500 Mouse, headset Corsair vengeance 1500
Software Win7 64bit Ultimate
Benchmark Scores 3d mark 2011: testing
i think Folding@Home complete all research soon
 
Joined
Jan 11, 2005
Messages
1,407 (0.24/day)
Location
66 feet from the ground
System Name 2nd AMD puppy
Processor FX-8350 vishera
Motherboard Gigabyte GA-970A-UD3
Cooling Cooler Master Hyper TX2
Memory 16 Gb DDR3:8GB Kingston HyperX Beast + 8Gb G.Skill Sniper(by courtesy of tabascosauz &TPU)
Video Card(s) Sapphire RX 580 Nitro+;1450/2000 Mhz
Storage SSD :840 pro 128 Gb;Iridium pro 240Gb ; HDD 2xWD-1Tb
Display(s) Benq XL2730Z 144 Hz freesync
Case NZXT 820 PHANTOM
Audio Device(s) Audigy SE with Logitech Z-5500
Power Supply Riotoro Enigma G2 850W
Mouse Razer copperhead / Gamdias zeus (by courtesy of sneekypeet & TPU)
Keyboard MS Sidewinder x4
Software win10 64bit ltsc
Benchmark Scores irrelevant for me
i think Folding@Home complete all research soon
i didn't hear any results like a cure or something and since 2000....

i'm skeptikal about this f@h do you know exactly what data is processed ?

Stanford say is protein folding and other molecular dynamics but is really this?
 
Joined
Oct 7, 2008
Messages
435 (0.10/day)
Location
United Arab Emirates
System Name The Fouth Light
Processor i7-2600K @ 3.80GHz (1.35voltage)
Motherboard GA-P67A-UD4-B3
Cooling Corsair H50
Memory 8GB DDR3 Corsair Vengence 1600MHz (8-8-8-24)
Video Card(s) 2x GTX460 @ Stock
Storage Cosair GTX Neutron 120GB + 2xWD Carviar Black 640GB RAID0
Display(s) Samsung S27B970D 27 Inch
Case Thermaltake Element V Full tower
Audio Device(s) Asus Xonar Essense STX
Power Supply Corsair HX1000W
Software Windows 7 64bit Ultimate
Finally, someone who is folding not for showing off....
 
Joined
Mar 28, 2007
Messages
2,490 (0.50/day)
Location
Your house.
System Name Jupiter-2
Processor Intel i3-6100
Motherboard H170I-PLUS D3
Cooling Stock
Memory 8GB Mushkin DDR3L-1600
Video Card(s) EVGA GTX 1050ti
Storage 512GB Corsair SSD
Display(s) BENQ 24in
Case Lian Li PC-Q01B Mini ITX
Audio Device(s) Onboard
Power Supply Corsair 450W
Mouse Logitech Trackball
Keyboard Custom bamboo job
Software Win 10 Pro
Benchmark Scores Finished Super PI on legendary mode in only 13 hours.
I made a post a couple of weeks ago about how I control and implement all of the clients. I think it's kinda slick even if I do say so myself, but necessity is the mother of invention. Controlling all the clients individually got old very, very fast. I had to think of a better way. You can see my post about that here.

Thanks for the interest guys,

Jason
Welcome -- I think it's really cool what you're doing to help out your Dad. Anytime anyone criticizes the amount of money it all costs, just remind them about that.

That being said, looks like I've found another admin who names a group of server units after characters in Atlas Shrugged. :p

i didn't hear any results like a cure or something and since 2000....

i'm skeptikal about this f@h do you know exactly what data is processed ?

Stanford say is protein folding and other molecular dynamics but is really this?
Ah, critics.

Cures take a long, long time to develop.
 

newtekie1

Semi-Retired Folder
Joined
Nov 22, 2005
Messages
27,403 (4.99/day)
Location
Indiana, USA
Processor Intel Core i7 10850K@5.2GHz
Motherboard AsRock Z470 Taichi
Cooling Corsair H115i Pro w/ Noctua NF-A14 Fans
Memory 32GB DDR4-3600
Video Card(s) ASUS Strix GTX 1080Ti
Storage 500GB SX8200 Pro + 8TB with 1TB SSD Cache
Display(s) QNIX QX2710 1440p@120Hz
Case Fractal Design Define S
Audio Device(s) Onboard is good enough for me
Power Supply eVGA SuperNOVA 1000w G3
Software Windows 10 Pro x64
@Atlas Folder: Are you also running SMP clients on the CPUs, or are you just doing pure GPU folding?
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,147 (5.90/day)
Location
IA, USA
System Name BY-2015
Processor Intel Core i7-6700K (4 x 4.00 GHz) w/ HT and Turbo on
Motherboard MSI Z170A GAMING M7
Cooling Scythe Kotetsu
Memory 2 x Kingston HyperX DDR4-2133 8 GiB
Video Card(s) Sapphire Radeon RX 5500 XT Pulse 8 GiB
Storage Crucial MX300 275 GB, Seagate Exos X12 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse SteelSeries Sensei RAW
Keyboard Tesoro Excalibur
Software Windows 10 Pro 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
I was expecting a custom, 23 PCI Express slot motherboard board loaded with GPUs (like a Tesla system). Needless to say, I'm disappointed. Anyway, that's quite a blatant waste of money and electricity, in my opinion.
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.77/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
what cpu are you using Atlas??
I wonder if it is i7s...
I think you can also run WCG while folding...
 

newtekie1

Semi-Retired Folder
Joined
Nov 22, 2005
Messages
27,403 (4.99/day)
Location
Indiana, USA
Processor Intel Core i7 10850K@5.2GHz
Motherboard AsRock Z470 Taichi
Cooling Corsair H115i Pro w/ Noctua NF-A14 Fans
Memory 32GB DDR4-3600
Video Card(s) ASUS Strix GTX 1080Ti
Storage 500GB SX8200 Pro + 8TB with 1TB SSD Cache
Display(s) QNIX QX2710 1440p@120Hz
Case Fractal Design Define S
Audio Device(s) Onboard is good enough for me
Power Supply eVGA SuperNOVA 1000w G3
Software Windows 10 Pro x64
what cpu are you using Atlas??
I wonder if it is i7s...
I think you can also run WCG while folding...
The boards he seems to be using looks like the MSI K9A2 Platinum, an AM2 board. So I would guess he is running some kind of Phenom.
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.77/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
damn! :(... I am still confused about how you link up all those pcs, and how do you get them all onto one monitor...?
 
Top