• We've upgraded our forums. Please post any issues/requests in this thread.

Video of the Day: 23 GeForce GTX 295 Video Cards Installed in a Single Server Rig

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (1.03/day)
Likes
299
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
#26
correction...folding power
 
Joined
Apr 7, 2009
Messages
4 (0.00/day)
Likes
5
Location
Tulsa, Oklahoma
#27
I|m sure that this guy has had a lot of fun building 6 machines into a 19 inch rack. But 6x 4 GTXs could be improved using a PCIe expansion bay. Since this is a folding farm and not GPU SLI, he doesnt need x16 slots, so an extension bay would be quite practical. He could use x4 for the cards, and easily drive 8 or 16 cards from ONE PC.
That is not correct. The nVidia driver only supports 8 GPUs. I have spent a lot of time researching this issue and at the current time it's not possible to use PCIe extenders or motherboards that have more than 4 PCIe slots for this reason. It is also not possible to run VMs on a machine with PCIe backplanes because there is no VM software yet that allows virtualization of the GPUs for CUDA. Xen is working on it though.

You can see a discussion here: http://foldingforum.org/viewtopic.php?f=38&t=9340&start=0

and here: http://foldingforum.org/viewtopic.php?f=38&t=9011

Jason
 
Joined
Jan 11, 2009
Messages
9,025 (2.77/day)
Likes
1,446
Location
Montreal, Canada
System Name Fun waste of money
Processor i7 3930k @ 4 Ghz
Motherboard AsRock X79 Fatal1ty Professional
Cooling Dark Rock Pro 3 (removed 120mm fan)
Memory 32GB (4x8gb) 2133Mhz CL9 Mushkin RedLine
Video Card(s) EVGA ACX 3.0 GTX 1060 6GB
Storage 3.25TB separated in 5 SSDs (SanDisk Extreme Pro, Intel 730 & 520, 2*Crucial MX100)
Display(s) 27" 2560x1440 [Korean] PCBANK PB2700
Case BeQuiet! SilentBase 800
Audio Device(s) iBasso D10 + AKG KXXX
Power Supply EVGA P2 750W 80+ Platinum semi-fanless
Mouse Sensei RAW
Keyboard Black cherry MX Thermaltake ESports
Software Windows 10 Pro X64
#29

mtosev

New Member
Joined
Mar 21, 2005
Messages
1,462 (0.31/day)
Likes
144
Location
Maribor, Slovenia
Processor Core i7 930
Motherboard Asus P6X58D Premium
Memory 6gb SuperTalent WB200UX6G8
Video Card(s) xfx radeon hd 5970 black edition
Storage Intel X25-M 80
Display(s) Dell UltraSharp 2209wa
Case Haf 932
Audio Device(s) Integreted sound card
Software w7
#30
HARDCORE :rockout:
 

sol.fides

New Member
Joined
Nov 21, 2005
Messages
65 (0.01/day)
Likes
1
Location
Merrit BC
System Name blimp
Processor E8600
Motherboard Asus P5K Deluxe
Cooling Scythe Infinity w/ Dual SFF21F fans
Memory 4gb ddr2-800 crucial balistix
Video Card(s) EVGA 8800GT (705*950)
Storage intel g2 80gb, x2 WD-640ls RAID0, WD-1002EX
Display(s) 2x Dell 2007WFP 20"
Case CM Stacker 830 w/ 4x140mm & 3x120mm
Audio Device(s) Soundblaster X-fi Fatal1ty FPS, Klipsch 2.1 & logitech 330
Power Supply Corsair HX1000
Software F@H 24/7
Benchmark Scores Lappy: g51vx
#31
The earth simulator had 41 teraflops of supercomputing power. it was #1 in the world starting in 2002 till 2004. It cost about 72 million dollars. congratulations on building an equivalent for under $30,000.
 
Joined
Sep 17, 2008
Messages
567 (0.17/day)
Likes
171
Location
Germany
#32
@ Atlas Folder

On daily basis, how much time do you spend controlling all your Clients? At the moment I have only 6 Clients and sometimes I get more than one down while I'm at work.
 
Joined
Jan 11, 2009
Messages
9,025 (2.77/day)
Likes
1,446
Location
Montreal, Canada
System Name Fun waste of money
Processor i7 3930k @ 4 Ghz
Motherboard AsRock X79 Fatal1ty Professional
Cooling Dark Rock Pro 3 (removed 120mm fan)
Memory 32GB (4x8gb) 2133Mhz CL9 Mushkin RedLine
Video Card(s) EVGA ACX 3.0 GTX 1060 6GB
Storage 3.25TB separated in 5 SSDs (SanDisk Extreme Pro, Intel 730 & 520, 2*Crucial MX100)
Display(s) 27" 2560x1440 [Korean] PCBANK PB2700
Case BeQuiet! SilentBase 800
Audio Device(s) iBasso D10 + AKG KXXX
Power Supply EVGA P2 750W 80+ Platinum semi-fanless
Mouse Sensei RAW
Keyboard Black cherry MX Thermaltake ESports
Software Windows 10 Pro X64
#33
I believe he isn't a TPU member... unfortunately
 
Joined
Apr 7, 2009
Messages
4 (0.00/day)
Likes
5
Location
Tulsa, Oklahoma
#34
ghost101 said:
What ppd do you get? Also whats the power usage like?
My 24 hour average is ~267,000 PPD. You can see that here.

The power consumption of a single shelf is ~1,125 Watts. I have 5 shelves and more misc equipment. Power consumption is probably around 7,000 Watts continuous. I haven't added it all up lately.

sol.fides said:
The earth simulator had 41 teraflops of supercomputing power. it was #1 in the world starting in 2002 till 2004. It cost about 72 million dollars. congratulations on building an equivalent for under $30,000.
Well thanks, but there is an important difference. True supercomputers use 64-bit floating point math (double-precision), Folding@Home makes do with 32-bit floating point math (single-precision). So yes, building supercomputers is getting cheaper fast, but there's still no free (cheap) lunch. If you need double-precision you still must pony up the cash.

mmaakk said:
On daily basis, how much time do you spend controlling all your Clients?
Not much... I can check for dead EUE clients anywhere from my iPhone by going here, it's updated every 60 seconds. If I need to make a correction all of the machines are controllable through LogMeIn and I have LogMeIn Ignition on my iPhone so I can log into the servers from my iPhone and tweak things as necessary.

I made a post a couple of weeks ago about how I control and implement all of the clients. I think it's kinda slick even if I do say so myself, but necessity is the mother of invention. Controlling all the clients individually got old very, very fast. I had to think of a better way. You can see my post about that here.

Thanks for the interest guys,

Jason
 
Joined
Aug 30, 2006
Messages
6,374 (1.55/day)
Likes
983
System Name ICE-QUAD // ICE-CRUNCH
Processor Q6600 // 2x Xeon 5472
Memory 2GB DDR // 8GB FB-DIMM
Video Card(s) HD3850-AGP // FireGL 3400
Display(s) 2 x Samsung 204Ts = 3200x1200
Audio Device(s) Audigy 2
Software Windows Server 2003 R2 as a Workstation now migrated to W10 with regrets.
#35
Thanks for coming to TPU and sharing your experience Jason. Great build(s), nice rack. I can see that perhaps the multiple PC route is easier than the extension bay option.

Did I understand you are driver-limited. Therefore max 8 GPUs per PC, so you would need a minimum of 3 independent PCs to run your 24 cards? Or does the x2 thing knock that down to just 4 cards (8 GPU dies)?

PS. Write to nVidia, show them your stuff, get them to fix the drivers, and sponsor your rack.
 
Joined
Apr 7, 2008
Messages
632 (0.18/day)
Likes
64
Location
Australia
System Name _Speedforce_ (Successor to Strike-X, 4LI3NBR33D-H, Core-iH7 & Nemesis-H)
Processor Intel Core i9 7900X (Lapped) @ 4.9Ghz With XSPC Raystorm (Lapped)
Motherboard Asus Prime X299 Deluxe (XSPC Watercooled) - Custom Heatsinks
Cooling XSPC Custom Water Cooling + Custom Air Cooling (From Delta 120's TFB1212GHE to Spal 30101504&5)
Memory 8x 8Gb Corsair Dominator Platinum 3400MHz @ 3667Mhz (CMU32GX4M4C3466C16)
Video Card(s) 3x Asus GTX1080 Ti (Lapped) With Customised EK Waterblock (Lapped) + Custom heatsinks (Lapped)
Storage 5x Samsung 960 Pro 1Tb M.2 2280 (Hyper M.2 x16 Card), 1x Samsung 850 Pro 1Tb, 6x Samsung EVO 850 4Tb
Display(s) 6x Asus ROG Swift PG27AQ
Case Aerocool Strike X (Modified)
Audio Device(s) Creative Sound BlasterX AE-5 & Aurvana XFi Headphones
Power Supply 2x Corsair AX1500i With Custom Sheilding, Custom Switching Unit. Braided Cables.
Mouse Razer Copperhead + R.A.T 9
Keyboard Ideazon Zboard + Optimus Maximus. Logitech G13.
Software w10 Pro x64.
Benchmark Scores pppft, gotta see it to believe it. . .
#36
The Atlas Folder, very nice folding setup.
I wonder what it will actually be crunching.
 

DaMulta

My stars went supernova
Joined
Aug 3, 2006
Messages
16,159 (3.89/day)
Likes
1,416
Location
Oklahoma T-Town
System Name Work in progress
Processor AMD 955---4Ghz
Motherboard MSi GD70
Cooling OcZ Phase/water
Memory Crucial2GB kit (1GBx2), Ballistix 240-pin DIMM, DDR3 PC3-16000
Video Card(s) CrossfireX 2 X HD 4890 1GB OCed to 1000Mhz
Storage SSD 64GB
Display(s) Envision 24'' 1920x1200
Case Using the desk ATM
Audio Device(s) Sucky onboard for now :(
Power Supply 1000W TruePower Quattro
#38
He was short a few cards on the bottom and it really kills the look of the system:p



Just kidding, and that's one hell of a folding rig you have!
 

HammerON

The Watchful Moderator
Staff member
Joined
Mar 2, 2009
Messages
7,434 (2.32/day)
Likes
4,740
Location
Up North
System Name Cruncher / Cruncher 2
Processor i7 6900K / E5 2683 v3
Motherboard X99A Gaming Pro Carbon / ASRock X99 Extreme 4/3.1
Cooling EK Supremacy EVO Elite 2011-3 / EK Supremacy
Memory G.Skill Trident Z 32GB 3200 / G.SKILL Trident Z 16GB 3200
Video Card(s) EVGA 980 Ti Classified / ATI 3650
Storage Samsung 960 Pro 512 GB + Mushkin Reactor 1 TB / OCZ Vertex 4 256 GB
Display(s) Dell UltraSharp U3011 30"
Case Corsair Obsidian 800D / TT Core V51
Audio Device(s) On-board
Power Supply Corsair AX1200 / EVGA 500W
Software Win 10 Pro/ Win 10
Benchmark Scores Always changing~
#39
Welcome to TPU Atlas :)
It is neat to see all that awsome hardware (especially the GPUs) working hard to find a cure :toast:
For those that want to learn more about his cause, check out the link to his website at bottom of his sig.
 
Joined
Apr 21, 2008
Messages
5,230 (1.48/day)
Likes
661
Location
IRAQ-Baghdad
System Name MASTER
Processor Core i7 3930k run at 4.4ghz
Motherboard Asus Rampage IV extreme
Cooling Corsair H100i
Memory 4x4G kingston hyperx beast 2400mhz
Video Card(s) 2X EVGA GTX680
Storage 2X Crusial M4 256g raid0, 1TbWD g, 2x500 WD B
Display(s) Samsung 27' 1080P LED 3D monitior 2ms
Case CoolerMaster Chosmos II
Audio Device(s) Creative sound blaster X-FI Titanum champion,Creative speakers 7.1 T7900
Power Supply Corsair 1200i, Logitch G500 Mouse, headset Corsair vengeance 1500
Software Win7 64bit Ultimate
Benchmark Scores 3d mark 2011: testing
#41
i think Folding@Home complete all research soon
 
Joined
Jan 11, 2005
Messages
1,088 (0.23/day)
Likes
360
Location
66 feet from the ground
System Name 2nd AMD puppy
Processor Phenom II x4 BE 970-3.6 Ghz
Motherboard Gigabyte GA-970A-UD3
Cooling Cooler Master Hyper TX2
Memory 16 Gb DDR3:8GB Kingston HyperX Beast + 8Gb G.Skill Sniper(by courtesy of tabascosauz &TPU)
Video Card(s) MSI HD6850 Cyclone;860/1100 Mhz
Storage SSD-840 pro 128 Gb;Seagate 500Gb;WD-1Tb
Display(s) Samsung 2032BW
Case NZXT 820 PHANTOM
Audio Device(s) Audigy SE with Logitech Z-5500
Power Supply Cooleer Master RP M520
Mouse Razer copperhead / Gamdias zeus (by courtesy of sneekypeet & TPU)
Keyboard MS Sidewinder x4
Software win7 64bit sp1
Benchmark Scores irrelevant for me
#42
i think Folding@Home complete all research soon
i didn't hear any results like a cure or something and since 2000....

i'm skeptikal about this f@h do you know exactly what data is processed ?

Stanford say is protein folding and other molecular dynamics but is really this?
 
Joined
Oct 7, 2008
Messages
432 (0.13/day)
Likes
43
Location
United Arab Emirates
System Name The Fouth Light
Processor i7-2600K @ 3.80GHz (1.35voltage)
Motherboard GA-P67A-UD4-B3
Cooling Corsair H50
Memory 8GB DDR3 Corsair Vengence 1600MHz (8-8-8-24)
Video Card(s) 2x GTX460 @ Stock
Storage Cosair GTX Neutron 120GB + 2xWD Carviar Black 640GB RAID0
Display(s) Samsung S27B970D 27 Inch
Case Thermaltake Element V Full tower
Audio Device(s) Asus Xonar Essense STX
Power Supply Corsair HX1000W
Software Windows 7 64bit Ultimate
#43
Finally, someone who is folding not for showing off....
 
Joined
Mar 28, 2007
Messages
2,481 (0.63/day)
Likes
340
Location
Your house.
System Name Jupiter-2
Processor Intel i3-6100
Motherboard H170I-PLUS D3
Cooling Stock
Memory 8GB Mushkin DDR3L-1600
Video Card(s) EVGA GTX 1050ti
Storage 512GB Corsair SSD
Display(s) BENQ 24in
Case Lian Li PC-Q01B Mini ITX
Audio Device(s) Onboard
Power Supply Corsair 450W
Mouse Logitech Trackball
Keyboard Custom bamboo job
Software Win 10 Pro
Benchmark Scores Finished Super PI on legendary mode in only 13 hours.
#44
I made a post a couple of weeks ago about how I control and implement all of the clients. I think it's kinda slick even if I do say so myself, but necessity is the mother of invention. Controlling all the clients individually got old very, very fast. I had to think of a better way. You can see my post about that here.

Thanks for the interest guys,

Jason
Welcome -- I think it's really cool what you're doing to help out your Dad. Anytime anyone criticizes the amount of money it all costs, just remind them about that.

That being said, looks like I've found another admin who names a group of server units after characters in Atlas Shrugged. :p

i didn't hear any results like a cure or something and since 2000....

i'm skeptikal about this f@h do you know exactly what data is processed ?

Stanford say is protein folding and other molecular dynamics but is really this?
Ah, critics.

Cures take a long, long time to develop.
 

newtekie1

Semi-Retired Folder
Joined
Nov 22, 2005
Messages
24,277 (5.51/day)
Likes
10,367
Location
Indiana, USA
Processor Intel Core i7 4790K@4.6GHz
Motherboard AsRock Z97 Extreme6
Cooling Corsair H100i
Memory 32GB Corsair DDR3-1866 9-10-9-27
Video Card(s) ASUS GTX960 STRIX @ 1500/1900
Storage 480GB Crucial MX200 + 2TB Seagate Solid State Hybrid Drive with 128GB OCZ Synapse SSD Cache
Display(s) QNIX QX2710 1440p@120Hz
Case Corsair 650D Black
Audio Device(s) Onboard is good enough for me
Power Supply Corsair HX850
Software Windows 10 Pro x64
#45
@Atlas Folder: Are you also running SMP clients on the CPUs, or are you just doing pure GPU folding?
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
20,914 (6.24/day)
Likes
10,009
Location
IA, USA
System Name BY-2015
Processor Intel Core i7-6700K (4 x 4.00 GHz) w/ HT and Turbo on
Motherboard MSI Z170A GAMING M7
Cooling Scythe Kotetsu
Memory 2 x Kingston HyperX DDR4-2133 8 GiB
Video Card(s) PowerColor PCS+ 390 8 GiB DVI + HDMI
Storage Crucial MX300 275 GB, Seagate 6 TB 7200 RPM
Display(s) Samsung SyncMaster T240 24" LCD (1920x1200 HDMI) + Samsung SyncMaster 906BW 19" LCD (1440x900 DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay
Audio Device(s) Realtek Onboard, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse SteelSeries Sensei RAW
Keyboard Tesoro Excalibur
Software Windows 10 Pro 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
#47
I was expecting a custom, 23 PCI Express slot motherboard board loaded with GPUs (like a Tesla system). Needless to say, I'm disappointed. Anyway, that's quite a blatant waste of money and electricity, in my opinion.
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (1.03/day)
Likes
299
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
#48
what cpu are you using Atlas??
I wonder if it is i7s...
I think you can also run WCG while folding...
 

newtekie1

Semi-Retired Folder
Joined
Nov 22, 2005
Messages
24,277 (5.51/day)
Likes
10,367
Location
Indiana, USA
Processor Intel Core i7 4790K@4.6GHz
Motherboard AsRock Z97 Extreme6
Cooling Corsair H100i
Memory 32GB Corsair DDR3-1866 9-10-9-27
Video Card(s) ASUS GTX960 STRIX @ 1500/1900
Storage 480GB Crucial MX200 + 2TB Seagate Solid State Hybrid Drive with 128GB OCZ Synapse SSD Cache
Display(s) QNIX QX2710 1440p@120Hz
Case Corsair 650D Black
Audio Device(s) Onboard is good enough for me
Power Supply Corsair HX850
Software Windows 10 Pro x64
#49
what cpu are you using Atlas??
I wonder if it is i7s...
I think you can also run WCG while folding...
The boards he seems to be using looks like the MSI K9A2 Platinum, an AM2 board. So I would guess he is running some kind of Phenom.
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (1.03/day)
Likes
299
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
#50
damn! :(... I am still confused about how you link up all those pcs, and how do you get them all onto one monitor...?