Server Project

[TopHatProductions115]

Hello there! I've hinted toward this project in various forum posts, but I'll cut to the chase. I'm in the final phase of a long-awaited server project, which began back in 2017/2018. However, I never got to actually acquire any of the parts until 2019/2020. With the massive gap in time, I had to change things up a bit, due to market forces and parts availability. The server is meant to replace (and extend) my current workstation - a Dell Precision T7500. This is what I've ended up with as of the 2020/2021 transition:


CSE :: HPE ProLiant DL580 G7
CPU :: 4x Intel Xeon E7-8870's (10c/20t each; 40c/80t total)
RAM :: 128GB (32x4GB) DDR3-1333 PC3-10600R ECC
STR :: 1x HP 518216-002 146GB HDD (ESXi, VMware Linux Appliance, System ISOs) +

1x 500GB Seagate Video ST500VT003 HDD (Remote Development VM) +​

4x HP 507127-B21 300GB HDDs +​

1x Western Digital WD Blue 3D NAND 500GB SSD (Virtual Flash) +​

1x Intel 320 Series SSDSA2CW600G3 600GB SSD (VFF)​

1x LSI SAS 9201-16e HBA SAS card (4-HDD DAS) +​

1x Mini-SAS SFF-8088 to SATA Forward Breakout x4 cable +​

1x Kingwin MKS-435TL (4x 3.5in HDD cage) +​

4x IBM Storwize V7000 98Y3241 4TB HDDs​

PCIe :: 1x HP 512843-001/591196-001 System I/O board +

1x HP 588137-B21; 591205-001/591204-001 PCIe Riser board​

GPU :: 1x nVIDIA GeForce GTX 1080 +

1x nVIDIA GRID K520​

SFX :: 1x Creative Sound Blaster Audigy Rx
NIC :: 1x HPE NC524SFP (489892-B21)
I/O :: 1x HPE PCIe ioDuo MLC I/O Accelerator (641255-001)
FAN :: 4x Arctic F9 PWM 92mm fans *
PSU :: 4x 1200W Server PSU's (HP 441830-001/438203-001)
PRP :: 1x Dell MS819 Wired Mouse
ODD :: 1x Sony Optiarc BluRay drive


Parts marked with * are already in-house, but require further planning/modification before they can be added to the server.

As of now, the fans aren't really required for functionality. They were meant to help quiet the server down a bit, but they require some modification to work. This part can wait.


Here is the current software configuration plan for the server:

• VMware Appliance (for managing and monitoring ESXi 6.5)
• Temporary/Test VM (for when I want to host a temporary role/application, without impacting mission-critical infrastructure)
• Windows Server Datacentre (Active Directory^, Technitium^, NTP^, SoftEther+, hMailServer+, AOMEI PXE Boot^, etc.)
• MacOS Server (PleX Server**, DaVinci Resolve, Google Drive/MEGA sync*, Handbrake, SVP4 Pro, iTunes, F@H, etc.)
• Artix Linux (OpenSSH^, strongSwan+, NextCloud** +, FreePBX+SMS+, ejabberd+, CUPS^, Server JRE, Minecraft servers, etc.)
• Windows 10 LTSB (front-end tasks - OBS Studio, Moonlight**, DesignWorks, Console Game Emulators, modpack dev., etc.)
• Remote Development (my build environment - Netbeans, MinGW/64, cmake, GNUWin, CUDA SDK, Mosh, nginx+SQLite+Php, etc.) ^

* Temporary task that will be replaced by a permanent, self-hosted solution
** Can benefit from port forwarding, but will be primarily tunnel-bound
^ Tunnel-bound (VPN/SSH) role - not port forwarded/exposed to the Internet
+ Active Directory enabled - Single Sign On (SSO)


May end up adding YouPHPTube to the Linux VM, depending on how the 4K livestreaming transition goes...
If you want more details about the project posted here, please let me know.


There are four other mirrors for this project, all of which get updated when new developments get made:

• LTT Forum
• L1 Forum
• Reddit post
• VMware Forum

 

[TopHatProductions115]

Replaced the Rosewill RASA-11001 with a Kingwin MKS-435TL, due to the fact that it doesn't need molex and will also have a cleaner look to it. Forgot how to edit posts here, sorry if I could have added the info to the OP above. Also planning out a long-term Tesla K80 upgrade...

 

[TopHatProductions115]

Grabbed the wired mouse. Now back to the waiting game, to see when everything will arrive in the mail...

 

[phill]

Looking forward to seeing the outcome

Although I do wonder, if 4 1200w PSUs will be enough??...

 

[TopHatProductions115]

The new drive cage arrived via Amazon. There's only one item left that hasn't arrived in the mail yet - my new mouse...

 

[TopHatProductions115]

Virtual Flash looks like fun. I might actually look into it more when the time comes...

 

[TopHatProductions115]

Looking forward to seeing the outcome

Although I do wonder, if 4 1200w PSUs will be enough??...

Thank goodness it only needs one of the PSUsat a time! Otherwise, I'd be toast XD

 

[phill]

Thank goodness it only needs one of the PSUsat a time! Otherwise, I'd be toast XD

I know that feeling!!! I have a few servers here, boy they are power hungry!! Looking forward to seeing another picture filled update!!

 

[TopHatProductions115]

So, here's the current plan:

• Have ESXi run 24/7, in a low-power/on-demand state
• Have Windows Server 2016 (VPN+AD) setup for 24/7 low-power operation as well
• Have Artix Linux (Nextcloud) setup for 24/7 low-power operation too
• Have MacOS setup for Hybrid Sleep after 20 minutes' inactivity and setup for Wake-on-LAN
• Have Windows 10 setup for ^^^ as well, since it's not running anything mission critical
• Don't power down ESXi unless I want to lose access to over half my infrastructure :3

Notes:

• https://helgeklein.com/blog/2012/03/dead-simple-wake-on-lan-for-your-windows-server-at-home/

 

[phill]

I look forward to seeing this all up and running

 

[TopHatProductions115]

So close, yet so far:
https://communities.vmware.com/message/2980320#2980320

 

[phill]

Have you got spare server RAM you can change it out with to test @TopHatProductions115 ??

 

[TopHatProductions115]

Have you got spare server RAM you can change it out with to test @TopHatProductions115 ??

Yes. And I'm getting ready to run a memory diagnostic, to see which sticks may be failing.

 

[TopHatProductions115]

Have you got spare server RAM you can change it out with to test @TopHatProductions115 ??

Crisis averted. From the looks of it, the issue was caused by me running a slightly older version of ESXi 6.5 (U2). VMware's servers have been a bit unreliable for downloads recently, so I had to dig for a copy of ESXi 6.5 U3 on the internet. Now troubleshooting the next issue:

• https://communities.vmware.com/message/2980646#2980646

Also have to look into this at some point (even though the server will be mostly hiding behind a VPN):

• https://kb.vmware.com/s/article/55636

 

[TopHatProductions115]

I'm getting ready to reinstall ESXi 6.5 U3, since HPE's Smart Array assumed I was gonna 'RAID all da things' and lose over half my storage to sub-optimally configured RAID arrays that it made (one of which was in perpetual recovery and included an unsupported SSD). Had to manually configure each drive as a logical volume with RAID 0 fault tolerance, treating each drive as its own virtual array of one disk. I ended up RAIDing all the things anyway, by that logic.

 

[TopHatProductions115]

Now to address the SSD issue in the background, while I look at security patches and initial VM setup.

 

[TopHatProductions115]

Currently trying to make a new datastore via SSH. More info here:

• https://communities.vmware.com/message/2980777#2980777

 

[TopHatProductions115]

So close to the final stage:

• Failed to create VMFS datastore - Cannot change the host configuration.

 

[TopHatProductions115]

While I'm waiting on comments for the previous issue, and a few ISOs to upload to my server, I can start working on investigating this:

• https://kb.vmware.com/s/article/55636
• https://kb.vmware.com/s/article/55806
• https://www.vmware.com/security/advisories/VMSA-2018-0020.html
  • https://kb.vmware.com/s/article/56547
  • https://kb.vmware.com/s/article/56563
  • https://kb.vmware.com/s/article/56896

Gotta search for the patches through this page, by entering the details mentioned in the last 3 kb pages:

• https://my.vmware.com/group/vmware/patch#search

On a side note, also ran into this when setting up my first VM:

• http://www.yellow-bricks.com/2018/0...when-powering-on-vm-on-vsphere/#comment-77449

Reserve Memory beforehand, I guess

 

[TopHatProductions115]

Here to update the Reddit mirror's link - those expire every 6 months

• https://www.reddit.com/user/TopHatProductions115/comments/ijoeg0/project_personal_datacentre/

 

[phill]

Someone has been having fun!! Gotta love trouble shooting don't you?? I hope it all settles out soon for you!!

 

[TopHatProductions115]

Someone has been having fun!! Gotta love trouble shooting don't you?? I hope it all settles out soon for you!!

Yeah. Still privately working out how to get my HBA working in ESXi as well, before I send that issue to VMware's forums as well XD

It's been hectic...

 

[phill]

Yeah. Still privately working out how to get my HBA working in ESXi as well, before I send that issue to VMware's forums as well XD

It's been hectic...

Sometimes I do ask myself the question about all that I do at home, for home and I say, is it all worth the hassle sometimes

I hope you get it sorted out mate and can report back all of the great work you've been doing, oh and don't forget a load of pictures!!

 

[TopHatProductions115]

Oh, I will :3

Re: Failed to create VMFS datastore - Cannot change the host configuration.

Hello TopHat, I was asking that because is not available on the HTML5 Client. For configuring VFFS you need to login to vCenter using the Flash client (vSphere Web Client). This is the URL you have to access and you can use same credentials: https://vcenter_server_fqdn/vsphere-client

communities.vmware.com

 

[TopHatProductions115]

Sometimes I do ask myself the question about all that I do at home, for home and I say, is it all worth the hassle sometimes

I hope you get it sorted out mate and can report back all of the great work you've been doing, oh and don't forget a load of pictures!!

One step closer