Farmerjoe is a distributed rendering system for Blender, it does both frame based distribution and bucket based (single frame) distribution, it has a web gui and is fairly easy to set up IMHO :) I had tried to get Drqueue running and it worked great on linux but was less than easy to use on windows. There are a few other distributed rendering systems for blender but I wanted something I could run independantly of blender and seeing how I knew perl I hacked together this system to get distributed rendering running for Blender the way I wanted it to work.Why Another Network Rendering System?
I originally tried to get DrQueue working, and while a very good system is a bit of a mission to get installed for windows and after struggling with it I figured that rather wasting anymore time trying out other systems, I would just write one that did things the way I wanted.How is Farmerjoe Different?
Well, I hate having to install all sorts of stuff to get another render slave working so Farmerjoe is designed to have no install for render slaves. I also dont like having to install much for the server, and as I only really know how to program in perl I wrote the whole system in Perl as a (single script for the mostpart) and also compiled it to an exe version so you dont even have to install perl. Farmerjoe also is, I think one of the first distributed rendering systems for Blender to do both frame based and bucket rendering. (bucket rendering = rendering a single image in parts afaik)What Platforms does Farmerjoe Work On?
In theory any platform blender and perl and blender run on, but I have only run it on Linux, Windows and now OSX, to run on other platforms a bit of code would need changing, I can fo this but cannot compile it as I only have Lin/Win/OSX
There are Windows, Linux and OSX Binaries included.What do I need to get Farmerjoe running?
I use /farmerjoe on my linux machine, or c:\farmerjoe on windows or /Users/lobo/farmerjoe on OSX
Rightclick the farmerjoe directory and choose sharing. Enable sharing and allow network users to modify the files.
NOTE: I use SMB (Windows Filesharing) on my linux machines as nfs shares caused an error when having the master PC also a slave - something about could not create text file - I figure it is something due to the same blend file being used by more than one instance of blender. Also NFS has allot more security so SMB is easier to set up for many linux slaves (for me anyway) Below is how I configure my Linux machines this will no doubt vary per distribution but should help you get started (I use Ubuntu and Debian) On my linux master I in my /etc/samba/smb.conf
[farmerjoe] guest account = lobo force user = lobo delete readonly = yes public = yes path = /farmerjoe force group = lobo available = yes browseable = yes writable = yes
On OSX I just turn on windows filesharing in Sytem Preferences > Sharing which shares my home directory where I keep Farmerjoe.
You do not have to share the farmerjoe directory itself
you can put the farmerjoe directoy inside another shared directory. In this case
simply alter the path in Farmerjoe.conf to suit eg:
If "c:\shared files" was the share and the path to Farmerjoe was "c:\shared files\farmerjoe" in the conf you would write the 'jobs' and 'log's paths like this:
jobs = farmerjoe\jobs
Use the View>Map Network Drive option in explorer, browse to the shared farmerjoe directory, allocate a drive letter (I use r: for render) select reconnect at startup if you dont want to map the driver every time you reboot.
Here is the line I have in my /etc/fstab on my linux slaves
//<master server>/farmerjoe /render smbfs guest,uid=lobo,gid=lobo 0 0
which mounts the shared directory on /render when they boot up.
On OSX I use Go>Connect to Server.. in the finder, then type in: smb://<master server>/farmerjoe which ends up being mounted in /Volumes/farmerjoe
Set the master IP number and port you want to run the server on.
Set the Farmerjoe root directory which is the shared directory, so on my linux machine I use /render and on windows I map to drive r: and on OSX /Volumes/farmerjoe
The conf that comes with farmerjoe is set up to use a blender binary on the master server which I do so that I dont need to install blender on each slave which makes setting up slaves very easy, however you can install blender on each slave as long as the paths are the same for each platform, or if the blender binary is in your path you can simply use 'blender' instead of the full path.
If you have a standard location for Imagemagick's convert you should be ok with the paths in the conf file supplied. Linux is pretty standard on the path for Imagemagick and windows is in the path if you used the Imagemagick installer. If yours differs you probably know where it is :)
If you want to use bucket rendering you will need Imagemagick installed as farmerjoe uses Imagemagick's convert to composite the parts back into a single image.
You can get Imagemagick from here http://www.imagemagick.org, or if using linux you should be able to install using your package manager. you can either set the windows_composite & linux_composite to the full path of imagemagick's convert or simply 'convert' if its in your path.
Once you have everything configured you can run Farmerjoe. You need to run the master server which accepts jobs and manages the clients. Then run the client (slave) on each computer you want to render images.Notes:
To run Farmerjoe as a Master Server use this command
To run Farmerjoe as a client use this command
Now you should be able to sumbit jobs directly from Blender. To submit a job you need to run the farmerjoe_submit.py script within blender, either put it in your scripts directory and run from Scripts>Render>Farmerjoe Submit Render or load farmerjoe_submit.py into blender as a text file and use ALT+P to run it. You can find farmerjoe_submit.py in the bin directory.
Once you have renders submitted you might want to check on their progress, you can telnet to the master server using the same port as in the Farmerjoe.conf file or you can run this command Farmerjoe.xxx --appserver which will run a web server that will allow you to see jobs progress and delete jobs and reset tasks as well as see the slaves connected. To connect to Farmerjoe with a web browser use the URL http://<master server ip/host>:<master server port>
When the render is complete you can retrieve the frames from the jobs directory look for a folder with the name of the job you submitted, once you have copied the frames to a safe place you can use the web gui to delete the job which will also remove the job directory - be warned there is no confirmation it just blows it away.
Bucket renders always render as TARGA files as I am cheating when compositing them, rather than cutting out the rect from the full image I want I simply composite them and rely on the alpha chanel to do the work. I plan on doing this better in the future - possibly using ideasmans composite function fron his blenderfarm script :) and use blender to composite the image parts. The imagemagick's composite also seems to give different results on windows than on linux, the image is still composited but the background colour differs.
There are many small modifications I'd like to add, an auto refresh for the web interface, easier to read progress, more checking etc, however it works pretty well for what I need it for and hopefully others will find it useful to as is.
Id also like to get rid of all the command windows, maybe just have one that runs the other required processes or somthing - suggestions are welcome.
If you need help to get Farmerjoe farming email me <lobo.nz (at) gmail (dot) com>
go to Freenode IRC Servers in the #blenderchat or #blender channel I am there quite often.
If you want to modify and/or use the perl file rather than use the binaries you will need these extra perl modules
The rest should come standard with perl.
You also need to change these three lines in the farmerjoe_submit.py in the #variables Section if you want the slaves to use the Farmerjoe.pl script too
'linux_farmerjoe' :'Farmerjoe.linux', 'windows_farmerjoe' :'Farmerjoe.exe', 'osx_farmerjoe' :'Farmerjoe.osx'
So that they refer to the Farmerjoe.pl
'linux_farmerjoe' :'Farmerjoe.pl', 'windows_farmerjoe' :'Farmerjoe.pl', 'osx_farmerjoe' :'Farmerjoe.pl'
If you make changes or improvements let me know and please email me a copy so I can include it for others. Also if you have suggestions or trouble please email me - I will try to reply to all emails :) <lobo.nz (at) gmail (dot) com>
Thanks to ideasman I have used a small portion of his blenderfarm script which I ported to perl to save writing the rect calculations for bucket rendering from scratch. Also thanks to those who have given feedback on Blenderartists, Email and IRC
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
THE PROGRAM IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, BUT WITHOUT ANY WARRANTY. IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. IN NO EVENT WILL THE AUTHOR WILL BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.