Render farm....

[adrianx]

hello to all

I want to build a render farm for 3dmax / maya / blender

so I have 3 pc in one location and 3 in 3 remote locations

I dont personally design and render any frame/image, I simple what to join to a group of design (for the IT position)

from that I dont receive any money but I will try to help this friends

the pc are with 2/4 core and one with 6 core

the video cards are some 5570 amd, 9600GT, and one hd 6870hd. Any pc have 4 GB ram, hdd and internet connexion

the idea is to build this render farm and learn how this distributive system work and how to improve that


so I'm open to any idea

 

[Solaris17]

their are plug ins for some of those programs that you install on the slave PC's (install program on master) and the master PC sends them work. the plugins talk to each other.

 

[adrianx]

thanks for fast reply

are this plugins work over the internet or only in local network?

 

[Solaris17]

thanks for fast reply

are this plugins work over the internet or only in local network?

local i think.

 

[Deleted member 3]

Technically you could probably use them over the internet, just open up the right ports or create a VPN. Though I have no clue about the bandwidth usage, so it might or might not be a good idea.

 

[crazyeyesreaper]

for 3d rendering its not gonna be all that bad but getting all the individual frames and layers from each machine will be a pain in the ass depending on how the rendering is done,

for example if i put out a full 2560x1600 frame * 24 frames per second * x # of layers spread across multiple machines it would become a nightmare. local networks can transfer that data fast enough but the internet, yea.. not so lucky i highly doubt you can maintain or afford a fiber network across all machines when located in different locations as just a side thing to help out.

since well 60 seconds of animation at 24 frames per second = 1440 renders, now if your doing say a 3d film or the like thats more along the lines of 172800 frames for 2 hrs of animation, now try transfering that in raw file formats to be manipulated and then compiled in something like nuke or shake it would be a nightmare. for the average user.

 

[adrianx]

I will try today with maya with 2 pc if it works or not...

for the remote pc I will need for sure a variant of VPN

 

[Benetanegia]

What they said. You can definately do it. You can set up a VPN or open up some ports.

For 3dmax and Maya you can use BackBurner and third party renderers like Vray have their own distributed rendering feature. I never did it myself but I think you just type in the IP and the rendering server automatically finds the clients, if there are any. Just remember that you need a net license or otherwise a standalone license for each PC in the net.

for 3d rendering its not gonna be all that bad but getting all the individual frames and layers from each machine will be a pain in the ass depending on how the rendering is done,

for example if i put out a full 2560x1600 frame * 24 frames per second * x # of layers spread across multiple machines it would become a nightmare. local networks can transfer that data fast enough but the internet, yea.. not so lucky i highly doubt you can maintain or afford a fiber network across all machines when located in different locations as just a side thing to help out.

since well 60 seconds of animation at 24 frames per second = 1440 renders, now if your doing say a 3d film or the like thats more along the lines of 172800 frames for 2 hrs of animation, now try transfering that in raw file formats to be manipulated and then compiled in something like nuke or shake it would be a nightmare. for the average user.

I don't think it's such a problem. If the required files and the output folder all are shared I think the distribution rendering plugins take care of everything. I suppose they want a farm because they have very long rendering times, in which case rendering time probably outweights data transfering time or required bandwidth. The internet is pretty fast these days and in the worst case scenario, they could set up 2 smaller farms with the three PCs each and do it locally. But really I don't think you need to do that. I have performed a few renderings on my desktop where the data was on a laptop and over 802.11b (Bandwidth <<< 11 MB/s) and I saw no significant difference over having files local.

 

[adrianx]

so I have now 2 pc with maya and mental ray ... also maya sattelite...

but I try and I have no luck to make a network render to work, the pc are in same switch at 2 meter distance, the file sharing work and the ping is ok.

but dont render in network node...

 

[crazyeyesreaper]

What they said. You can definately do it. You can set up a VPN or open up some ports.

For 3dmax and Maya you can use BackBurner and third party renderers like Vray have their own distributed rendering feature. I never did it myself but I think you just type in the IP and the rendering server automatically finds the clients, if there are any. Just remember that you need a net license or otherwise a standalone license for each PC in the net.



I don't think it's such a problem. If the required files and the output folder all are shared I think the distribution rendering plugins take care of everything. I suppose they want a farm because they have very long rendering times, in which case rendering time probably outweights data transfering time or required bandwidth. The internet is pretty fast these days and in the worst case scenario, they could set up 2 smaller farms with the three PCs each and do it locally. But really I don't think you need to do that. I have performed a few renderings on my desktop where the data was on a laptop and over 802.11b (Bandwidth <<< 11 MB/s) and I saw no significant difference over having files local.

transferring between machines on the same network aka in a studio or the like isnt that bad but going from say home to office to lab in 3 different locations can be a nightmare.

 

[Benetanegia]

transferring between machines on the same network aka in a studio or the like isnt that bad but going from say home to office to lab in 3 different locations can be a nightmare.

If they set up a VPN and a file server I don't see how? All machines work from the same shared folders. And it seems it has that working already.

Anyway, considering the size and nature of the farm, I'm assuming they are a small group, not involved in huge projects. I'm also assuming they were already doing it in separate locations, based once again on the farm he pretends to make, so in any case it is an improvement right? At least that's how I see it.

Now let's see if we can help him, because I never did distributed rendering myself, the only knowledge I have is from reading the documentation of 3dsmax and mentalray several years ago.

so I have now 2 pc with maya and mental ray ... also maya sattelite...

but I try and I have no luck to make a network render to work, the pc are in same switch at 2 meter distance, the file sharing work and the ping is ok.

but dont render in network node...

Maybe you should try giving some more especific info so someone can help you sort out the problem. We don't know what you did or you didn't, what fails, etc...

 

[crazyeyesreaper]

yea ive only used distributed rendering a few times back in college, now i just manually setup machines with scene files and ahve them render set frames aka do frames 1-200 on machine 1 201-400 on machine 2 etc, it was quicker for me to do it that way then to do distributed especially since back then we were using Maya 8 which had issues with certain rigs and animated items where they would basically explode during render time destorying the frames and causing us to have to revert back to older Maya versions namely 7.0 which we had to then rename all scene files and items from MA files to work as MB files wont load in previous versions such a pain in the ass that was... anyway off topic

im pretty much out of my league on this, i dont remember nearly enough info to troubleshoot it.

The OPs best bet is to go to a forum like CG Talk, and ask there.

 

[adrianx]

so ...

something is not OK i can render locally but no jobs go to slave nodes (same situation from node pc)

 

[Solaris17]

are you using a VPN? or are you running on a LAN? check to make sure the correct ports are forwarded for each machine on their respective equipment.

 

[adrianx]

I resolve the problem ... was the config file location ... in C:\users\myuser\...

I also make same test and from start the cpu with 2 core are ... bad in big numbers

my conclusion was that the time needed to render a scene is very dependent by the processing power of the nodes. so if I have a slow machine the farm the rendering process will wait for that machine to end the job. also I dont have now how to set the size of the jobs for every machines from farm.

the scope of this render for a small amount of money (something like 50-100$ per 5-7 days)

my idea is that the client contact me and I received the job and start render and resend to client the render file/frame

because the my system are in different location (geographically) , I think that a VPN or an over-internet render will do the job...(for now I dont know any software that can do that)
I found a boinc client that do same render jobs and that work like any boinc client. But this network is not so free, also you need to get the job approved by a guy or two. for that the idea remain only one idea (not a practical one).

in the next day I will try a different software and will post here the results.

I'm daily on the internet, but I dont enter on the forum