Keiretsu | Homelab

[CheapMeat]

Hey all, I pretty much visit TPU every single day. It's my homepage actually too. So I figured aside from Reddit's r/homelab, this is the best place to talk about my builds with like minded people who also appreciate hardware.

Warning: This is not a guide, or a best practices build or reasonable at all. Most of my choices make no sense, especially financially. It's literally just me liking hardware for the sake of hardware and building whatever.

Background: I'm a huge fan of the Connection Machine (Thinking Machine), I even went to NYC just to see the CM-2. I'm also a big fan of Japanese 90s supercomputers & IBM mainframes. I love blinkenlights. I love the idea of having a machine that looks like it's plotting against me. So what I'm building is just an extension of something fun, that appeals to me.


Keiretsu

I feel like this name is the perfect name for my systems, the cooperation between them, the loose organization between them, etc. It also sounds badass to me, hah. https://en.wikipedia.org/wiki/Keiretsu

I have way more gear but so far I want to limit myself, which is already a lot, to 4 x 4U whiteboxes. Here are the names of my systems and some of the current specs. I became completely obsessed with the motherboards, and dual proc systems in general, which is why I have them even if complete overkill or even outdated at this point. They're seriously the main thing in my setup I built around. I was able to buy each with a bit of RAM and a mix of V3 and V4 CPUs-mid-range but I've been sidegrading the CPUs recently.

Tokyo: Router / Switch / Networking

Kyoto: Compute | Rosewill RSV-L4000 chassis, ASUS Z10PE-D16 WS, (soon) 2 x Xeon E5-2699 V4's CPUs

Yokohama: Storage | Rosewill RSV-L4500 chassis, ASUS Z10PE-D8 WS, downgraded to 2 x Xeon E5-2637 v4 CPUs

Nagoya: Storage | Rosewill RSV-L4500 chassis, ASUS Z10PE-D8 WS, downgraded to 2 x Xeon E5-2637 v4 CPUs

Osaka: Gaming | (soon) Rosewill RSV-L4000B, ASUS X99-WS/IPMI, Intel i7-6950X.


Software: I'm a Windows guy, I love a GUI, hah. So I'm using Windows 10 Pro for Workstations on all 4 (Pro is enough but I just wanted to be extra). I'm also using StableBit DrivePool (+ cloud & scanner), PrimoCache, Teracopy & ProcessLasso on all 4 systems. I'm also a big fan of Folding@Home and BOINC + Gridcoin. So I let my machines do that when not in direct usage. To manage all the computers, I use RDP mostly through Osaka (my main rig) or connect an extra keyboard, mouse & monitor around directly.


(photos: sorry about my dirty room!)

 

[CheapMeat]

I decided to save the 2U Silverstone chassis for later when the day comes to upgrade Osaka (gaming machine). I'd use that in the 2U then and turn the ASUS X99-WS/IPMI into a Router and name that Tokyo. Instead I'm moving it currently to a 4U Rosewill mining chassis. I really really love giant heatsinks on GPUs like the Morpheus from RAIJINTEK. And this chassis lets you put the GPUs in the front. So I'm going to buy the Camino PCIe extensions and put the GPUs in the front. This way I can maximize PCIe slots too instead of covering up slots without going watercooling.

Some goals for the next post: buy 5x more 80mm fans, get Osaka back up and running, start setting up the network drive pools and setup caching, finish installing the E5 V4 CPUs I have.

 

[CheapMeat]

Spent the day building Osaka, everything worked fine for 10 minutes and then the PSU popped and I smelled something burnt. Was an EVGA 750W bronze PSU. =/ I tried a different PSU and the motherboard seems to be dead now. Great. I know for sure it wasn't a power limit issue since I had no GPU installed other than a GT 710 and I had the same PSU powering it for over a year.

 

[phill]

Great to see all the pics and hardware, love seeing the hardware!! After all, projects forum, you need to see the money, I mean the hardware

Gutted to hear about the PSU Was the motherboard the X99 Asus??

 

[CheapMeat]

Hah, thank you for the comment. It was the ASUS X99-WS/IPMI https://www.asus.com/us/Commercial-Servers-Workstations/X99WSIPMI/. I've been trying to think if I should directly replace it or get something in the X299 range.

 

[CheapMeat]

I was trying to figure out StableBit DrivePools Cloud "File Share", so I could have just "one" shared network drive that I could drag and drop files into and it would then make copies on each system (Nagoya & Yokohama). But I simply can't seem to make that much sense of it at the moment. There also seems to be size limit. It seems powerful but maybe I'm just too dumb to understand it properly. So instead I'm using ChoEazyCopy to do this (which is a GUI ver. of Robocopy it seems, yay for me). This way I just drag files to say Nagoya and then it copies it to Yokohama....in theory.

So on Kyoto, I have Nagoya's and Yokohama's networked drive showing up. I use the Windows 10 network sharing settings to make it possible.

It basically goes like this:

Nagoya DrivePool => Nagoya Network Drive
Yokohama DrivePool => Yokohama Network Drive
Kyoto DrivePool => Kyoto Network Drive

Here's an example of how it looks.



I just also set up the SSD cache (Write cache) for the drives.

Here's how I'd love to have the entire setup:




I also got two new purchases. An old school but solid voltage regulator and an AMD Radeon Pro Duo Polaris!

 

[CheapMeat]

Installing these two on Yokohama today.

The CPU is 4C/8T. So now Yokohama will have a total 8C/16T. It's a downgrade from the original 2 x 20C/40T V3's but these have a 3.50Ghz base clock and 3.70 boost clock. I think that's plenty for the storage focused server. But also for some reason I was just drawn to this Xeon CPU to try out. Just something cool about it.

Then I'm installing my go-to heatsink. It not only fits awesomely in the 4U chassis, it's got RGB goodness (even if I have no window yet haha), has 4 heatpipes and can handle 160W TDP. It's just a nice aesthetically pleasing cooler overall. So far I have I think 5 of them. As you can tell, I do some stuff for the heck of it cool factor haha. Something about the lights make the CPUs look like reactor cores, and with a blinking effect, it has a blinkenlight "doing work" type of vibe, which I love.

Eventually I'm going to use Protocase to make Acrylic window tops for the 4U Rosewill cases, so I can see it all.

 

[phill]

I think I have read some good things on those Cryorig coolers Please let us know how you get on!!