• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Video of the Day: 23 GeForce GTX 295 Video Cards Installed in a Single Server Rig

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
correction...folding power
 

Atlas Folder

New Member
Joined
Apr 7, 2009
Messages
4 (0.00/day)
Location
Tulsa, Oklahoma
I|m sure that this guy has had a lot of fun building 6 machines into a 19 inch rack. But 6x 4 GTXs could be improved using a PCIe expansion bay. Since this is a folding farm and not GPU SLI, he doesnt need x16 slots, so an extension bay would be quite practical. He could use x4 for the cards, and easily drive 8 or 16 cards from ONE PC.
That is not correct. The nVidia driver only supports 8 GPUs. I have spent a lot of time researching this issue and at the current time it's not possible to use PCIe extenders or motherboards that have more than 4 PCIe slots for this reason. It is also not possible to run VMs on a machine with PCIe backplanes because there is no VM software yet that allows virtualization of the GPUs for CUDA. Xen is working on it though.

You can see a discussion here: http://foldingforum.org/viewtopic.php?f=38&t=9340&start=0

and here: http://foldingforum.org/viewtopic.php?f=38&t=9011

Jason
 
Joined
Jan 11, 2009
Messages
9,231 (1.66/day)
Location
Montreal, Canada
System Name Homelabs
Processor Ryzen 5900x | Ryzen 1920X
Motherboard Asus ProArt x570 Creator | AsRock X399 fatal1ty gaming
Cooling Silent Loop 2 280mm | Dark Rock Pro TR4
Memory 128GB (4x32gb) DDR4 3600Mhz | 128GB (8x16GB) DDR4 2933Mhz
Video Card(s) EVGA RTX 3080 | ASUS Strix GTX 970
Storage Optane 900p + NVMe | Optane 900p + 8TB SATA SSDs + 48TB HDDs
Display(s) Alienware AW3423dw QD-OLED | HP Omen 32 1440p
Case be quiet! Dark Base Pro 900 rev 2 | be quiet! Silent Base 800
Power Supply Corsair RM750x + sleeved cables| EVGA P2 750W
Mouse Razer Viper Ultimate (still has buttons on the right side, crucial as I'm a southpaw)
Keyboard Razer Huntsman Elite, Pro Type | Logitech G915 TKL
Joined
Mar 21, 2005
Messages
1,581 (0.23/day)
Location
Maribor, Slovenia, EU
System Name Core i9 rig / Lenovo laptop
Processor Core i9 10900X / Core i5 8350U
Motherboard Asus Prime X299 Edition 30 / Lenovo motherboard
Cooling Corsair H115i PRO RGB / stock cooler
Memory Gskill 4x8GB 3600mhz / 16GB 2400mhz
Video Card(s) Asus ROG Strix RTX 2080 Super / UHD 620
Storage Samsung SSD 970 PRO 1TB / Samsung OEM 256GB NVMe
Display(s) Dell UltraSharp UP3017 / Full HD IPS touch
Case Coolermaster mastercase H500M
Audio Device(s) Onboard sound
Power Supply Enermax Platimax 1700 watt / Lenovo 65watt power adapter
Mouse Logitech M500s
Keyboard Cherry
Software Windows 11 Pro / Windows 11 Pro

sol.fides

New Member
Joined
Nov 21, 2005
Messages
65 (0.01/day)
Location
Merrit BC
System Name blimp
Processor E8600
Motherboard Asus P5K Deluxe
Cooling Scythe Infinity w/ Dual SFF21F fans
Memory 4gb ddr2-800 crucial balistix
Video Card(s) EVGA 8800GT (705*950)
Storage intel g2 80gb, x2 WD-640ls RAID0, WD-1002EX
Display(s) 2x Dell 2007WFP 20"
Case CM Stacker 830 w/ 4x140mm & 3x120mm
Audio Device(s) Soundblaster X-fi Fatal1ty FPS, Klipsch 2.1 & logitech 330
Power Supply Corsair HX1000
Software F@H 24/7
Benchmark Scores Lappy: g51vx
The earth simulator had 41 teraflops of supercomputing power. it was #1 in the world starting in 2002 till 2004. It cost about 72 million dollars. congratulations on building an equivalent for under $30,000.
 
Joined
Sep 17, 2008
Messages
569 (0.10/day)
Location
Germany
@ Atlas Folder

On daily basis, how much time do you spend controlling all your Clients? At the moment I have only 6 Clients and sometimes I get more than one down while I'm at work.
 
Joined
Jan 11, 2009
Messages
9,231 (1.66/day)
Location
Montreal, Canada
System Name Homelabs
Processor Ryzen 5900x | Ryzen 1920X
Motherboard Asus ProArt x570 Creator | AsRock X399 fatal1ty gaming
Cooling Silent Loop 2 280mm | Dark Rock Pro TR4
Memory 128GB (4x32gb) DDR4 3600Mhz | 128GB (8x16GB) DDR4 2933Mhz
Video Card(s) EVGA RTX 3080 | ASUS Strix GTX 970
Storage Optane 900p + NVMe | Optane 900p + 8TB SATA SSDs + 48TB HDDs
Display(s) Alienware AW3423dw QD-OLED | HP Omen 32 1440p
Case be quiet! Dark Base Pro 900 rev 2 | be quiet! Silent Base 800
Power Supply Corsair RM750x + sleeved cables| EVGA P2 750W
Mouse Razer Viper Ultimate (still has buttons on the right side, crucial as I'm a southpaw)
Keyboard Razer Huntsman Elite, Pro Type | Logitech G915 TKL
I believe he isn't a TPU member... unfortunately
 

Atlas Folder

New Member
Joined
Apr 7, 2009
Messages
4 (0.00/day)
Location
Tulsa, Oklahoma
ghost101 said:
What ppd do you get? Also whats the power usage like?

My 24 hour average is ~267,000 PPD. You can see that here.

The power consumption of a single shelf is ~1,125 Watts. I have 5 shelves and more misc equipment. Power consumption is probably around 7,000 Watts continuous. I haven't added it all up lately.

sol.fides said:
The earth simulator had 41 teraflops of supercomputing power. it was #1 in the world starting in 2002 till 2004. It cost about 72 million dollars. congratulations on building an equivalent for under $30,000.

Well thanks, but there is an important difference. True supercomputers use 64-bit floating point math (double-precision), Folding@Home makes do with 32-bit floating point math (single-precision). So yes, building supercomputers is getting cheaper fast, but there's still no free (cheap) lunch. If you need double-precision you still must pony up the cash.

mmaakk said:
On daily basis, how much time do you spend controlling all your Clients?
Not much... I can check for dead EUE clients anywhere from my iPhone by going here, it's updated every 60 seconds. If I need to make a correction all of the machines are controllable through LogMeIn and I have LogMeIn Ignition on my iPhone so I can log into the servers from my iPhone and tweak things as necessary.

I made a post a couple of weeks ago about how I control and implement all of the clients. I think it's kinda slick even if I do say so myself, but necessity is the mother of invention. Controlling all the clients individually got old very, very fast. I had to think of a better way. You can see my post about that here.

Thanks for the interest guys,

Jason
 
Joined
Aug 30, 2006
Messages
7,195 (1.12/day)
System Name ICE-QUAD // ICE-CRUNCH
Processor Q6600 // 2x Xeon 5472
Memory 2GB DDR // 8GB FB-DIMM
Video Card(s) HD3850-AGP // FireGL 3400
Display(s) 2 x Samsung 204Ts = 3200x1200
Audio Device(s) Audigy 2
Software Windows Server 2003 R2 as a Workstation now migrated to W10 with regrets.
Thanks for coming to TPU and sharing your experience Jason. Great build(s), nice rack. I can see that perhaps the multiple PC route is easier than the extension bay option.

Did I understand you are driver-limited. Therefore max 8 GPUs per PC, so you would need a minimum of 3 independent PCs to run your 24 cards? Or does the x2 thing knock that down to just 4 cards (8 GPU dies)?

PS. Write to nVidia, show them your stuff, get them to fix the drivers, and sponsor your rack.
 
Joined
Apr 7, 2008
Messages
633 (0.11/day)
Location
Australia
System Name _Speedforce_ (Successor to Strike-X, 4LI3NBR33D-H, Core-iH7 & Nemesis-H)
Processor Intel Core i9 7980XE (Lapped) @ 5.2Ghz With XSPC Raystorm (Lapped)
Motherboard Asus Rampage VI Extreme (XSPC Watercooled) - Custom Heatsinks (Lapped)
Cooling XSPC Custom Water Cooling + Custom Air Cooling (From Delta 220's TFB1212GHE to Spal 30101504&5)
Memory 8x 8Gb G.Skill Trident Z RGB 4266MHz @ 4667Mhz (2x F4-4266C17Q-32GTZR)
Video Card(s) 3x Asus GTX1080 Ti (Lapped) With Customised EK Waterblock (Lapped) + Custom heatsinks (Lapped)
Storage 1x Samsung 970 EVO 2TB - 2280 (Hyper M.2 x16 Card), 7x Samsung 860 Pro 4Tb
Display(s) 6x Asus ROG Swift PG348Q
Case Aerocool Strike X (Modified)
Audio Device(s) Creative Sound BlasterX AE-5 & Aurvana XFi Headphones
Power Supply 2x Corsair AX1500i With Custom Sheilding, Custom Switching Unit. Braided Cables.
Mouse Razer Copperhead + R.A.T 9
Keyboard Ideazon Zboard + Optimus Maximus. Logitech G13.
Software w10 Pro x64.
Benchmark Scores pppft, gotta see it to believe it. . .
The Atlas Folder, very nice folding setup.
I wonder what it will actually be crunching.
 

DaMulta

My stars went supernova
Joined
Aug 3, 2006
Messages
16,168 (2.51/day)
Location
Oklahoma T-Town
System Name Work in progress
Processor AMD 955---4Ghz
Motherboard MSi GD70
Cooling OcZ Phase/water
Memory Crucial2GB kit (1GBx2), Ballistix 240-pin DIMM, DDR3 PC3-16000
Video Card(s) CrossfireX 2 X HD 4890 1GB OCed to 1000Mhz
Storage SSD 64GB
Display(s) Envision 24'' 1920x1200
Case Using the desk ATM
Audio Device(s) Sucky onboard for now :(
Power Supply 1000W TruePower Quattro
He was short a few cards on the bottom and it really kills the look of the system:p



Just kidding, and that's one hell of a folding rig you have!
 

HammerON

The Watchful Moderator
Staff member
Joined
Mar 2, 2009
Messages
8,397 (1.52/day)
Location
Up North
System Name Threadripper
Processor 3960X
Motherboard ASUS ROG Strix TRX40-XE
Cooling XSPC Raystorm Neo (sTR4) Water Block
Memory G. Skill Trident Z Neo 64 GB 3600
Video Card(s) PNY RTX 4090
Storage Samsung 960 Pro 512 GB + WD Black SN850 1TB
Display(s) Dell 32" Curved Gaming Monitor (S3220DGF)
Case Corsair 5000D Airflow
Audio Device(s) On-board
Power Supply EVGA SuperNOVA 1000 G5
Mouse Roccat Kone Pure
Keyboard Corsair K70
Software Win 10 Pro
Benchmark Scores Always changing~
Welcome to TPU Atlas :)
It is neat to see all that awsome hardware (especially the GPUs) working hard to find a cure :toast:
For those that want to learn more about his cause, check out the link to his website at bottom of his sig.
 
Joined
Apr 21, 2008
Messages
5,250 (0.90/day)
Location
IRAQ-Baghdad
System Name MASTER
Processor Core i7 3930k run at 4.4ghz
Motherboard Asus Rampage IV extreme
Cooling Corsair H100i
Memory 4x4G kingston hyperx beast 2400mhz
Video Card(s) 2X EVGA GTX680
Storage 2X Crusial M4 256g raid0, 1TbWD g, 2x500 WD B
Display(s) Samsung 27' 1080P LED 3D monitior 2ms
Case CoolerMaster Chosmos II
Audio Device(s) Creative sound blaster X-FI Titanum champion,Creative speakers 7.1 T7900
Power Supply Corsair 1200i, Logitch G500 Mouse, headset Corsair vengeance 1500
Software Win7 64bit Ultimate
Benchmark Scores 3d mark 2011: testing
i think Folding@Home complete all research soon
 
Joined
Jan 11, 2005
Messages
1,491 (0.21/day)
Location
66 feet from the ground
System Name 2nd AMD puppy
Processor FX-8350 vishera
Motherboard Gigabyte GA-970A-UD3
Cooling Cooler Master Hyper TX2
Memory 16 Gb DDR3:8GB Kingston HyperX Beast + 8Gb G.Skill Sniper(by courtesy of tabascosauz &TPU)
Video Card(s) Sapphire RX 580 Nitro+;1450/2000 Mhz
Storage SSD :840 pro 128 Gb;Iridium pro 240Gb ; HDD 2xWD-1Tb
Display(s) Benq XL2730Z 144 Hz freesync
Case NZXT 820 PHANTOM
Audio Device(s) Audigy SE with Logitech Z-5500
Power Supply Riotoro Enigma G2 850W
Mouse Razer copperhead / Gamdias zeus (by courtesy of sneekypeet & TPU)
Keyboard MS Sidewinder x4
Software win10 64bit ltsc
Benchmark Scores irrelevant for me
i think Folding@Home complete all research soon

i didn't hear any results like a cure or something and since 2000....

i'm skeptikal about this f@h do you know exactly what data is processed ?

Stanford say is protein folding and other molecular dynamics but is really this?
 
Joined
Oct 7, 2008
Messages
435 (0.08/day)
Location
United Arab Emirates
System Name The Fouth Light
Processor i7-2600K @ 3.80GHz (1.35voltage)
Motherboard GA-P67A-UD4-B3
Cooling Corsair H50
Memory 8GB DDR3 Corsair Vengence 1600MHz (8-8-8-24)
Video Card(s) 2x GTX460 @ Stock
Storage Cosair GTX Neutron 120GB + 2xWD Carviar Black 640GB RAID0
Display(s) Samsung S27B970D 27 Inch
Case Thermaltake Element V Full tower
Audio Device(s) Asus Xonar Essense STX
Power Supply Corsair HX1000W
Software Windows 7 64bit Ultimate
Finally, someone who is folding not for showing off....
 
Joined
Mar 28, 2007
Messages
2,490 (0.40/day)
Location
Your house.
System Name Jupiter-2
Processor Intel i3-6100
Motherboard H170I-PLUS D3
Cooling Stock
Memory 8GB Mushkin DDR3L-1600
Video Card(s) EVGA GTX 1050ti
Storage 512GB Corsair SSD
Display(s) BENQ 24in
Case Lian Li PC-Q01B Mini ITX
Audio Device(s) Onboard
Power Supply Corsair 450W
Mouse Logitech Trackball
Keyboard Custom bamboo job
Software Win 10 Pro
Benchmark Scores Finished Super PI on legendary mode in only 13 hours.
I made a post a couple of weeks ago about how I control and implement all of the clients. I think it's kinda slick even if I do say so myself, but necessity is the mother of invention. Controlling all the clients individually got old very, very fast. I had to think of a better way. You can see my post about that here.

Thanks for the interest guys,

Jason

Welcome -- I think it's really cool what you're doing to help out your Dad. Anytime anyone criticizes the amount of money it all costs, just remind them about that.

That being said, looks like I've found another admin who names a group of server units after characters in Atlas Shrugged. :p

i didn't hear any results like a cure or something and since 2000....

i'm skeptikal about this f@h do you know exactly what data is processed ?

Stanford say is protein folding and other molecular dynamics but is really this?

Ah, critics.

Cures take a long, long time to develop.
 

newtekie1

Semi-Retired Folder
Joined
Nov 22, 2005
Messages
28,472 (4.25/day)
Location
Indiana, USA
Processor Intel Core i7 10850K@5.2GHz
Motherboard AsRock Z470 Taichi
Cooling Corsair H115i Pro w/ Noctua NF-A14 Fans
Memory 32GB DDR4-3600
Video Card(s) RTX 2070 Super
Storage 500GB SX8200 Pro + 8TB with 1TB SSD Cache
Display(s) Acer Nitro VG280K 4K 28"
Case Fractal Design Define S
Audio Device(s) Onboard is good enough for me
Power Supply eVGA SuperNOVA 1000w G3
Software Windows 10 Pro x64
@Atlas Folder: Are you also running SMP clients on the CPUs, or are you just doing pure GPU folding?
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,259 (4.65/day)
Location
IA, USA
System Name BY-2021
Processor AMD Ryzen 7 5800X (65w eco profile)
Motherboard MSI B550 Gaming Plus
Cooling Scythe Mugen (rev 5)
Memory 2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s) AMD Radeon RX 7900 XT
Storage Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse Nixeus REVEL-X
Keyboard Tesoro Excalibur
Software Windows 10 Home 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
I was expecting a custom, 23 PCI Express slot motherboard board loaded with GPUs (like a Tesla system). Needless to say, I'm disappointed. Anyway, that's quite a blatant waste of money and electricity, in my opinion.
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
what cpu are you using Atlas??
I wonder if it is i7s...
I think you can also run WCG while folding...
 

newtekie1

Semi-Retired Folder
Joined
Nov 22, 2005
Messages
28,472 (4.25/day)
Location
Indiana, USA
Processor Intel Core i7 10850K@5.2GHz
Motherboard AsRock Z470 Taichi
Cooling Corsair H115i Pro w/ Noctua NF-A14 Fans
Memory 32GB DDR4-3600
Video Card(s) RTX 2070 Super
Storage 500GB SX8200 Pro + 8TB with 1TB SSD Cache
Display(s) Acer Nitro VG280K 4K 28"
Case Fractal Design Define S
Audio Device(s) Onboard is good enough for me
Power Supply eVGA SuperNOVA 1000w G3
Software Windows 10 Pro x64
what cpu are you using Atlas??
I wonder if it is i7s...
I think you can also run WCG while folding...

The boards he seems to be using looks like the MSI K9A2 Platinum, an AM2 board. So I would guess he is running some kind of Phenom.
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
damn! :(... I am still confused about how you link up all those pcs, and how do you get them all onto one monitor...?
 
Top