Never done WCG but would like to.

[Hellfire]

Hi guys, so never done WCG however I know the concept well, With the fact I am getting solar power installed at home I figured I could make most of the power to help out WCG and the TPU team?

I am decommissioning the following from work and figured they'd make good WCG crunchers? let me know your thoughts.

Two x HP Proliant DL360 G9's with the following specs
- 2 x Xeon 8 Core 16 Thread (32 total threads at 2.5Ghz)
- 128Gb Ram
- 12TB of SAS HDD (15k)

Not sure if they'd have much of an impact but figured the systems could work well? Welcome your thoughts.

 

[thebluebumblebee]

Are you thinking about only using them for crunching? If so:
Crunching does not use much ram.
Those HDD's would use a fair amount of watts, I'd get a small cheap SSD or 7200 rpm laptop HDD instead.
Most of use would recommend using Linux.

I think @phill is using some similar equipment, also with solar.

To get started with WCG, just use this link: https://join.worldcommunitygrid.org?teamId=S8TLJ6TFV1

 

[Hellfire]

Are you thinking about only using them for crunching? If so:
Crunching does not use much ram.
Those HDD's would use a fair amount of watts, I'd get a small cheap SSD or 7200 rpm laptop HDD instead.
Most of use would recommend using Linux.

I think @phill is using some similar equipment, also with solar.

To get started with WCG, just use this link: https://join.worldcommunitygrid.org?teamId=S8TLJ6TFV1

Cheers,

Honestly, got nothing else that I can use them for, either that or sell them maybe.

Not going to use them for much else except a media server (hence the disks) but that'll be running on another VM, I'd have 2 VM's set up for Crunching, probably allocate a small amount of drive space and minimal ram with 28 of the threads on one and 32 on the other, so I reckon maybe 60 threads of computing power across both racks.

 

[phill]

Hi guys, so never done WCG however I know the concept well, With the fact I am getting solar power installed at home I figured I could make most of the power to help out WCG and the TPU team?

I am decommissioning the following from work and figured they'd make good WCG crunchers? let me know your thoughts.

Two x HP Proliant DL360 G9's with the following specs
- 2 x Xeon 8 Core 16 Thread (32 total threads at 2.5Ghz)
- 128Gb Ram
- 12TB of SAS HDD (15k)

Not sure if they'd have much of an impact but figured the systems could work well? Welcome your thoughts.

Hey @Hellfire

@thebluebumblebee is correct, I use old servers from work to crunch away, be careful tho, they really can chew through the electric lol The ones I have are Dell R710's, I use them with two L5640's as they are lower power CPUs they might take a little longer to get work units done, but as I always believe, it's not about doing this fast just efficiently When the server is running and the WCG is running on all 24 threads, I believe the server can use up about 275w without too much fuss. This one has 5 1Tb drives, SAS external card and dual PSUs (which you use more power running together as I'm sure you know) My SR-2 which uses X5650's at stock speeds, that uses up about the 310w but the CPU clock speeds are slightly different, 2.26Ghz for the L5640's and 2.67Ghz for the X5650's. I've not overclocked them but the system is fully water cooled

The specs of the server look amazing I'm hoping to grab an upgrade of a server or two at some point as I'm looking at the R720 which would be Sandy Bridge Xeon's I believe, so hopefully more efficient. I think the CPUs they use aren't clocked high but again, just work well

Masses of storage and those 15k spinners will be amazing for fast access - Question, are the drive bays in the DL360's, 2.5" or 3.5"? If they where smaller drives, using 2 for an OS or VMware or whatever then the rest for storage if you so desire
If I leave the R710 on 24/7, I believe I can average about 10k - 12k a day with it, depending on work units My 5960X @ 4.2Ghz averages a little higher than it and at slightly lower watts. I'm sad cos I test what each server takes at idle, load etc, just gives me an idea of how much it would cost to run if I wasn't using solar Answer is, a lot!

Are you thinking about only using them for crunching? If so:
Crunching does not use much ram.
Those HDD's would use a fair amount of watts, I'd get a small cheap SSD or 7200 rpm laptop HDD instead.
Most of use would recommend using Linux.

I think @phill is using some similar equipment, also with solar.

To get started with WCG, just use this link: https://join.worldcommunitygrid.org?teamId=S8TLJ6TFV1

Cheers,

Honestly, got nothing else that I can use them for, either that or sell them maybe.

Not going to use them for much else except a media server (hence the disks) but that'll be running on another VM, I'd have 2 VM's set up for Crunching, probably allocate a small amount of drive space and minimal ram with 28 of the threads on one and 32 on the other, so I reckon maybe 60 threads of computing power across both racks.

I'm with @thebluebumblebee on the ram side of things too, as long as you have 1Gb per thread and seeing you have 128Gb of ram in each server, I believe your covered I agree also with the HDs, if you can get away with smaller/slower drives, it does really make a difference..

I would consider putting the servers in maybe a garage where the temp would be quite low, as I'd guess inside the house, they might get warm.. I need to do some testing with some other servers I've had as with 7 going at once, I think it might get too hot and simply not work.. I'll have to find out

What do you use for VM's? We use VMWare at work but I believe the free copy only supports a single CPU (however many cores though) but on all of my desktops I have running with it, I just put Linux on there and add TeamViewer to access them. I hook them up to monitors, as then you can get the full 1080P res and not 640 x 480 or whatever it might be.. That's painful to use!!
With 60 threads working away, you'd have a good few point each day If you choose specific work units, that will also boost scores but I tend to just do them all and don't worry about my scores as such..

As for the solar, I have a 3.6kW array, works so very well in the summer months, can make up to 30kW's a day which is quite hard to use up! It'll peak about the 4kW mark, so this is why I try to run more during the day, sadly though it's hotter then too so kind of a right pain

I really hope I've not waffled on too much, if there's anything else you need to know or have questions on please just ask

 

[Hellfire]

Hey bud,

Thanks for the reply. These are dual hot swap PSU's. One thing I have been considering is not bothering with the media server and leaving them hooked up and running in my office at work with my dev equipment. (Free power) then I can have them crunching to my heart's content.

As for which VM software we use Vsphere, the good thing is these two servers will come with the licenses so no need to worry about them. (Migrating to AWS)

I'll know more once I have written the plans to decommission them. I've not checked what CPUs they are as not jumped on Vsphere for them.

I'm surprised your other PC 5960x does more than the servers.

 

[phill]

No worries at all @Hellfire

They crunch fairly well but for the likes of a few Ryzen 1700/1700X etc etc, they pump out about half the performance but use about twice the power (compared to these R710's with dual L5640's in, they also have dual PSUs in which uses a little more power again on top, so can get away not using them if you like )

I'm using the same thing at home, I have a friend that works in VMWare, which recommended a few things so I'm looking forward to trying it

I've got a slightly slower CPU that was also given to me from work, saved them thousands on the licencing so without doubt I brought it home!! 8 core, 16 thread 2.0Ghz CPU so nothing massively fancy but it'll do all I need it to when I can get a R720 home with me

The 5960X can overclock further but running at 4.20Ghz I'm not really pushing it too hard as it's barely over 1v on the core I think it's just the efficiencies of the CPU and such that make it that bit faster. X58 has still got the power, just not the efficiency sadly... I think in the models you have are fairly new aren't they? They really are great power houses

What is it you do @Hellfire for work?

 

[Hellfire]

No worries at all @Hellfire

They crunch fairly well but for the likes of a few Ryzen 1700/1700X etc etc, they pump out about half the performance but use about twice the power (compared to these R710's with dual L5640's in, they also have dual PSUs in which uses a little more power again on top, so can get away not using them if you like )

I'm using the same thing at home, I have a friend that works in VMWare, which recommended a few things so I'm looking forward to trying it

I've got a slightly slower CPU that was also given to me from work, saved them thousands on the licencing so without doubt I brought it home!! 8 core, 16 thread 2.0Ghz CPU so nothing massively fancy but it'll do all I need it to when I can get a R720 home with me

The 5960X can overclock further but running at 4.20Ghz I'm not really pushing it too hard as it's barely over 1v on the core I think it's just the efficiencies of the CPU and such that make it that bit faster. X58 has still got the power, just not the efficiency sadly... I think in the models you have are fairly new aren't they? They really are great power houses

What is it you do @Hellfire for work?

Yeah these are literally current gen, bought from HPE maybe 12 months ago and still in warranty.

I'm an IT Business Support Manager, so run the IT Dept and all our external providers.

 

[Sasqui]

2 x Xeon 8 Core 16 Thread (32 total threads at 2.5Ghz)

I feel the onset on an inferiority complex lol. You will rack up points fast. Like others mentioned, ram isn't hit that hard, it's all about threads and gHz.

Might be worth mentioning, but you'll get a bump in points per day using Linux, it's more efficient than windows. Both of my WCG systems are running Ubuntu desktop.

Do it!

Question about your solar, do you have "net metering" there?

 

[phill]

Yeah these are literally current gen, bought from HPE maybe 12 months ago and still in warranty.

I'm an IT Business Support Manager, so run the IT Dept and all our external providers.

They'll be very efficient then in comparison to my servers here It would be interesting to hear what power it uses if you do get to take them home

The Xeon I have here that I'm waiting for another is a E5-2640 V2

I've not long started in IT, just over 14 months ago.. Been at home doing it for 20+ years but this is a different league, but I'm very much enjoying it

@Sasqui - What do you mean by net metering for the solar?

 

[Sasqui]

@Sasqui - What do you mean by net metering for the solar?

Here in the US, most states require the excess solar kWh power from residential (and commercial too usually) get pushed back to the energy grid for others to buy. The solar energy customer gets a 1:1 credit* for the energy put back to the grid. With old mechanical power meters, they would literally spin backwards, hence the term "net" metering. A power customer who generated more energy than used would literally get paid for the excess, or at least credited.

*the 1:1 credit varies depending on the state.

 

[Hellfire]

Hey Sasqui, We have something here,

We get paid a generation fee, for every K/W we generate, (3.86p per KW) even if we use this we get paid for it just for generating.

Then we get paid for exporting back to the grid, however there is no accurate way for the power being exported to be monitored so the government pays out a blanket value of 50% of what you generate is exported, (so even if I use every KW of power I generate they will pay me 50% as an exported fee (5.24p per KW)

So it can make great financial sense for solar PV installs here.

 

[phill]

I think @Hellfire has answered perfectly fine but there's differences in what I get to what he's getting (which isn't surprising, it's the UK!! lol)

The amounts might be different depending on when you have the solar installed and sign up to the tariff, but he's right in that you have a payment for what you produce and then 50% of that again as it's based on what goes back to the grid

My basic way of doing it is, use everything you possibly can as you'll get paid for it regardless unless you go on to a tariff were you have to give readings of what is exported and they'll work it out. The 50/50 is a lot easier, nicer and depending on the size of your array and what you use, you might find that you earn more but use more electric at the same time

The solar installs have definitely come down in price but this is why the price of the tariffs have dropped as well. It sucks really but it's the way it's worked as some people earn hundreds a quarter on top, I'm not even close to a £100 But then the UK isn't the sunniest of places.....

 

[Hellfire]

@phill what size system have you got, I'm getting a 4.5 KW/h system put in very soon and using the data from PVGIS estimates over 4300 KW for me over the year.

 

[phill]

@Hellfire - So many apologises, I must have missed this reply!!

I have a 3.6kW system, but it can produce up to 4kW (near as...) due to the panels I have and the fact it seems to be over producing about 10% at the best of it.. I've kept tabs on the amount I produce (I know sad but would like to compare year on year for any dips or noticeable decreases...)

I only had enough space on my roof for 12 panels and I think there's a limit for 'homes' or residential properties at about 16 or 18 panels I think?? You must have a big roof!!

Last year I saw just over 4000 units being produced, which I thought was pretty decent but it was a very sunny summer... At my best I've been producing 30kW a day in the summer.. It's crazy

I had a bunch of numpties installing mine, so I hope you have better luck! Do you know of the inverter and panels that you're having put in??

 

[Hellfire]

15x Longi 300w panels
Growatt 3600 inverter
Geo Solo generation meter

I got a smart meter that saves the daily history and generation so it'll perform the history for me all the time. We have a biggish house so lots of good roof space.

 

[phill]

@Hellfire I think they will need to put in a bigger inverter as 3600 isn't going to be enough for the panels you have...

Had the same issues with the company that did mine.. Put a 3.6kW system in that would push to 4kW, yet they put in a 2.8kW inverter.. Yeah they knew what they where doing!! I also had the Geo Solo generation meter and what a waste of £400 that was.. It's in my actic at the moment doing nothing because that's how good it was!! lol

Judging by the size of the array, you could well be needing closer to a 5kW inverter and I believe they might need to actually do something (I apologise if I have the terminology wrong) but it might need something called dual string configuration.. (I have a single string configuration on my array) They split the array into two so if one of the panels drops off in voltage it doesn't take the rest with them.. I think I have that correct.. It's been a while since I'd need do anything with it.. But might be worth an ask or a look into in Google

Panels

Geo Solo 3, shouldn't have bothered with it!!

It used to get to about this much production..... It never worked as it should have, I was told two different stories, but never had any luck returning the unit...

So when I was producing about 3kW, this happened...

Unfortunately with all the pictures that I did take, I never took one of the side of the small inverter but the model number was a Growatt 3000TL, specs here... Growatt 3000TL Inverter Specs

One of the tech's explaining to me why they installed what they did...

"Regarding the inverter readings and output, it is a rule through MCS to always undersize the inverter to keep it running at full potential and highest efficiency so the inverter installed is perfectly fine for the system and will not be hindering the production levels at all. The inverters work up to 20% over the maximum readings and the array itself will work up to a maximum of 90% meaning all readings will be within range to keep to high efficiency. "

Here came the new one...

Bit better No faulting inverter this time!!
The new basic solar monitoring system

A basic solar monitor.. About £40 ish at the time.. I then went on to buy the ShineLink, which was about £80 I think, but in the end that gave up the ghost as well (I'm guessing that the Growatt products are a little flakey but hopefully others might have better luck!!) as each few days, the unit would unsync and stop reading the inverter.. Rather frustrating as it would then not work for 3 days until it was disconnected completely and then just had to plug it back in and away it would go.. When it worked, it was amazing, but when it spends more time out of the unit than in it, I just give up.. Ended up sending that one back and getting a refund, but had a free one sent to me by Growatt, that was the same.. The guys who sold it too me where brilliant (not the solar installers they where fricking useless...) but Midsummer Energy, where very helpful and even refunded me without any issues... Very recommended

Overall, with the quality of install from the company they where awful but the quality of the solar panels, I can't fault it. It works and when the sun is out, free electric is amazing, hence running so much for the WCG, I couldn't afford it otherwise. I'm cutting back a little since the weather has been so poor of late, that I've had to keep a few of the bigger more inefficient PC's off... I can't afford a big electric bill, even more so with the girl friend stopping work now to have our first child in February... It's already tight as it is with my wage cut from my new IT job But still, I digress....

I hope this is of some use to you

 

[Hellfire]

Cheers for the help.

Lots of good info, I agree the 4500w system would need a bigger inverter but they're on two roofs, one South facing and one West facing. Because of this the system can not possibly peak over the 3600w of the inverter. If it was on one roof I would agree but my max production at any one time will be 3.247 KW/H according to PVGIS.

Do you generate more than 3.6Kw at any one time, in my research National grid states anyone producing more than 3.68KW/H output requires a DNO application made to the National Grid for the feed in.

 

[phill]

Ah, then that makes a little more sense If on one roof you could still split the panels into two areas, just depends on how you have them setup Hopefully better than I did!!

I have been known to peak over 3.6Kw at times but it doesn't hold it there for very long that I'm aware of. I'm guessing by the fact that the installed system is only 3.6Kw, then no I haven't. I do like the feed in tarriffs, but my god they are very poor compared to what they used to be.. A friend at work makes nearly 5 to 6 times what I do and he's on smaller panels and a slightly smaller system. But then his install cost him £12k, mine with the Geo 3 meter, was installed for under £5k. If I hadn't had the monitor unit, it would have been nearer £4.5k