Render Farm

[Killer_Rubber_Ducky]

I am looking at building a render farm for use with Cinelerra (which requires Linux). I want to spend the least I can but still get the most bang for buck. i am looking at like maybe 5 systems chained together. i want to loan it to my University's Broadcasting Lab for use with their FinalCut Pro lab. I used to be a Broadcasting student and the amount of time i spent rendering was sad especially when there was a mistake and i had to start all over again. Please help me select the proper hardware. REMEMBER, it needs to be Linux compatible hardware wand work with cinelerra as well. http://cinelerra.org/


I dont want to go above $2k

[Kreij]

So ... $400 per system? That seems a little skinny to me for rendering machines.

[Killer_Rubber_Ducky]

Well i did say maybe 5 systems (or less) i can go more $ but i dont want to have to. Remember i will be using something like Ubuntu or Fedora as the OS so knock off the cost of MicroShaft. Basically im looking for a good spec out to go by for the first clients and then be able to expand at a later date with identical systems.

[toastem2004]

not easy, but doable. also it look like that program can take advantage of nvidia gpus to help render and encode. Here my recommendation:

AMD Phenom II x3 740
MSI Mobo
4GB DDR3 Ram
nvidia GT430 Video Card
1TB Samsung HDD

all in all, a decent system for ~$356 us, all from the egg. All ya need is to grab a cooler for CPU and a cheap ol case and PS. Bit over $400 after all those, but considering ur budget, its still quite a lot of system for the price.

[(FIH) The Don]

sorry i am NO expert in this

but you can build 2 i7 systems for 2k, wouldnt that be better?
then you would have 16 threads to work with

[Killer_Rubber_Ducky]

Originally Posted by toastem2004

not easy, but doable. also it look like that program can take advantage of nvidia gpus to help render and encode. Here my recommendation:

AMD Phenom II x3 740
MSI Mobo
4GB DDR3 Ram
nvidia GT430 Video Card
1TB Samsung HDD

all in all, a decent system for ~$356 us, all from the egg. All ya need is to grab a cooler for CPU and a cheap ol case and PS. Bit over $400 after all those, but considering ur budget, its still quite a lot of system for the price.

Thanks, How would that do in say a rackmount setup or an small box setup ergo shuttle etc? Although, a suggestion for the gfx card, since this is a render farm, why not the http://www.newegg.com/Product/Produc...82E16814133354 and the mobo had some questionable reviews when I looked up the mobo with ubuntu in the search.

Originally Posted by (FIH) The Don

sorry i am NO expert in this

but you can build 2 i7 systems for 2k, wouldnt that be better?
then you would have 16 threads to work with

You would have a virtual 16 threads not physical. When it comes to rendering, physical is better. besides, a render farm node should be at least 4 machines. While
intel supports linux well for the most part, i need to be able to convince my school that using Linux for the rendering is a cost effective choice.

[toastem2004]

Rackmount, no idea. heat wise its pretty modest, so it should be ok in a small enclosure, but then you would need to sub the mobo for a mATX as the one i selected was a full ATX. I do know that most the gt430's are low profile so ur good on that as well.

However, i too am no expert at this as well. i toy with linux, but dont really put it to use like it should (excpt as a sudo windows update server)
As for the quadro card, its essentially the same thing, just double the price. from what i read on the software that ur using, it wouldnt make a difference in your application. CAD/Maya on windows, yes, it does make quite a difference (drivers mainly) but i cant see any reason to spend that kinda cash for ur project.

And in regards to the mobo with ubuntu, no idea. When i do use linux i use Mandrivia / PCLinuxOS. I can tell you from personal experiance that i haven't had any issue with the mATX board i listed above in the 2 distros i've used on it. YMMV though...

[Killer_Rubber_Ducky]

Originally Posted by toastem2004

Rackmount, no idea. heat wise its pretty modest, so it should be ok in a small enclosure, but then you would need to sub the mobo for a mATX as the one i selected was a full ATX. I do know that most the gt430's are low profile so ur good on that as well.

However, i too am no expert at this as well. i toy with linux, but dont really put it to use like it should (excpt as a sudo windows update server)
As for the quadro card, its essentially the same thing, just double the price. from what i read on the software that ur using, it wouldnt make a difference in your application. CAD/Maya on windows, yes, it does make quite a difference (drivers mainly) but i cant see any reason to spend that kinda cash for ur project.

And in regards to the mobo with ubuntu, no idea. When i do use linux i use Mandrivia / PCLinuxOS. I can tell you from personal experiance that i haven't had any issue with the mATX board i listed above in the 2 distros i've used on it. YMMV though...

Well, I was looking at it because it has 96 CUDA cores and cinelerra should be able to do GPU rendering and use CUDA. but i degress, im more of an ati guy these days. i got out of nvidia after my 7600gt

[toastem2004]

well the gt430 is also a 96 "cuda" core, and is cuda compliant as are all current nvidia desktop chips. also the gt430 runs those cores @ 700/1400 (core/shaders). And from what i read on cinelerra, it can use the nvidia cards to render/encode/ accelerate the video. All it says is via OpenGL, but it says stuff bout nvidia, not ati, so nvidia it is.

ps- generaly i'm an ati guy as well, but gotta go with what works best, in this case nvidia is the clear winner

[Killer_Rubber_Ducky]

Originally Posted by toastem2004

well the gt430 is also a 96 "cuda" core, and is cuda compliant as are all current nvidia desktop chips. also the gt430 runs those cores @ 700/1400 (core/shaders). And from what i read on cinelerra, it can use the nvidia cards to render/encode/ accelerate the video. All it says is via OpenGL, but it says stuff bout nvidia, not ati, so nvidia it is.

ps- generaly i'm an ati guy as well, but gotta go with what works best, in this case nvidia is the clear winner

Cool, thanks for explaining it to me. When I looked at the specs for the 430 i didnt see the CUDA cores. Any Idea what version of OpenGL? Im probably going to compile it for 64 bit if they dont have a debian or RPM binary for it already. If I go with Fedora I will try it that way. I havent had many good "experiences" with fedora as opposed to ubuntu/debian based distros.
I also need to figure out the sound and networking. to do a render farm Im going to need to hook them up on a intranet with Gigabit to minimize latency. the 4 "slaves" will be tied to the "Master" system via tcp port 400.

[toastem2004]

the gt430 supports opengl 4.1 / directx 11 and opencl. As for the compiling/network/audio... i'm sadly gonna have to bow out as that is well beyond my knowledge. Hopefully someone else here can be of assistance to you on that end of things.

[angelkiller]

Render farms are fascinate me. I'm by no means an expert though. Far from it. Anyway.

Linux is more cost effective because you don't have any Windows licnses in your 2k budget. Each box needs a ~$100 license so with 4 machines, 20% of your budget is already gone for the software.

Now does your 2k budget include the host machine? Typical render farms are set up so you do work from one computer (like the actual editing) then send the work to the farm to be rendered.

But having a seperate host isn't really necessary. Lets say you bulid four boxes, one of them can just be the host. Hopefully Cinelerra can use the host machine to render when set to use multiple nodes.

Hardware wise, I think you need to get as much CPU power as possible. I think RAM capacity comes in a close second and hard drive throughput is third. I would priortize in that order. (all imo)

The Phenom X6 may actually be the right processor to use here. It's cheap and has a bunch of cores. Platform wise, AMD tends to also cheaper compared to Intel. I would go this route. 1055T. I would overclock it as far as it can comfortably go on stock voltage. Err on the conservative side though. Stability is paramount. (but milk stability for all it's worth!)

I'd get the stardard 4GB (2x2GB) of DDR3. Speed is relatively unimportant. I'm not sure how much RAM each node needs. But 4GB seems like alot for doing just one task. Hopefully that should be enough or close to it. The cost of upgrading to 8GB may be worth looking into. Maybe not initially though because adding an additional 4GB to the machines will be a drop in solution.

The motherboard is a place to cheap out on. Not as in buying crap, but you don't need fancy motherboards. As far a requirements, they need to support the X6, 4GB of RAM and have gigabit ethernet. That's pretty much it. Maybe a cheap 785G motherboard? One of the most important things I would keep in mind when chosing is upgradability. The ability to just drop in RAM at a later date is good. So is the ability to support newer processors. That one is harder to judge, but it's something to look into. So in the future, you won't have to get a new CPU and motherboard for each node, you just drop a new CPU in.

Each node doesn't necessarily need local storage. But each node does need to boot an OS, so I guess the simplest solution would be to give each node some kind of hard drive. All the drive will be used for is to boot the OS. The host computer needs a decently fast storage array. How fast? I'm just not sure. The capacity of this array also depends on your needs. But a Raid 0+1 array of 1TB drives seems like a decent place to start.

As far as GPUs go, I would bother with it. Cinelerra's documentation is old. When it says it can use nVidia's cards to speed up rendering, it gives an example of a 7600GS. This is pre-CUDA. So I don't think Cinelerra uses GPUs for the actual rendering. Sounds like it uses the GPU to help with playback or something. But I'm seeing little evidence that Cinelerra supports CUDA or something similar.

Power is something to consider. The easiest solution is to use 1 PSU per node. Maybe a cheap PSU from a reliable brand.

Also an enclosure will probably be necessary. Honestly, I would just skip the enclosure and do something like this. (dammit can't find the link I'm looking for. Basically you take four really long (2-3ft) screws and you use nuts to stack the motherboards on top of each other. Makes a nice tower.) But if an enclosure is necessary, I would cheap out on these. Rackmount seems wayyy out of budget. Get some cheap mATX cases. You could save even more money by using a case that comes with a PSU. The built in PSU is crappy for sure, but budget is important and there's little risk of putting other components at risk. I would THINK that if one of these PSUs fails it won't destroy anything.

Everything I've said are my comments and opinions that I've developed from prior knowledge as well as a bit of reading through Cinelerra's documentation. Don't take what I say as final. They're all just suggestions. Hope this helps.

Interesting Links:

• Cinelerra documentation
• Info about Cinelerra and farms
• Helmer
• A recent Tom's article on render farms