Farmerjoe - The render Farmer

A distributed Rendering system for Blender

Homepage http://blender.formworks.co.nz

TOC

  1. Introduction
  2. Installation
  3. Starting
  4. TODO
  5. Help
  6. Using Perl instead of Binaries
  7. Thanks
  8. License
  9. DISCLAIMER

Introduction ^

Farmerjoe is a distributed rendering system for Blender, it does both frame based distribution and bucket based (single frame) distribution, it has a web gui and is fairly easy to set up IMHO :) I had tried to get Drqueue running and it worked great on linux but was less than easy to use on windows. There are a few other distributed rendering systems for blender but I wanted something I could run independantly of blender and seeing how I knew perl I hacked together this system to get distributed rendering running for Blender the way I wanted it to work.

Why Another Network Rendering System?

I originally tried to get DrQueue working, and while a very good system is a bit of a mission to get installed for windows and after struggling with it I figured that rather wasting anymore time trying out other systems, I would just write one that did things the way I wanted.

How is Farmerjoe Different?

Well, I hate having to install all sorts of stuff to get another render slave working so Farmerjoe is designed to have no install for render slaves. I also dont like having to install much for the server, and as I only really know how to program in perl I wrote the whole system in Perl as a (single script for the mostpart) and also compiled it to an exe version so you dont even have to install perl. Farmerjoe also is, I think one of the first distributed rendering systems for Blender to do both frame based and bucket rendering. (bucket rendering = rendering a single image in parts afaik)

What Platforms does Farmerjoe Work On?

In theory any platform blender and perl and blender run on, but I have only run it on Linux, Windows and now OSX, to run on other platforms a bit of code would need changing, I can fo this but cannot compile it as I only have Lin/Win/OSX

There are Windows, Linux and OSX Binaries included.

What do I need to get Farmerjoe running?

Farmerjoe (http://blender.formworks.co.nz)
Blender (http://www.blender.org)
Imagemagick (http://www.imagemagick.org)

Installation ^

You dont so much install Farmerjoe but configure him :)
  1. Make a directory for him

    I use /farmerjoe on my linux machine, or c:\farmerjoe on windows or /Users/lobo/farmerjoe on OSX

  2. Share the directory you just made so its available on your network.

  3. Mount shares on the slaves

  4. Edit the configuration file Farmerjoe.conf
  5. Set up Imagemagick

    If you want to use bucket rendering you will need Imagemagick installed as farmerjoe uses Imagemagick's convert to composite the parts back into a single image.

    You can get Imagemagick from here http://www.imagemagick.org, or if using linux you should be able to install using your package manager. you can either set the windows_composite & linux_composite to the full path of imagemagick's convert or simply 'convert' if its in your path.

Starting ^

Once you have everything configured you can run Farmerjoe. You need to run the master server which accepts jobs and manages the clients. Then run the client (slave) on each computer you want to render images.

Notes:

To run Farmerjoe as a Master Server use this command

To run Farmerjoe as a client use this command

Now you should be able to sumbit jobs directly from Blender. To submit a job you need to run the farmerjoe_submit.py script within blender, either put it in your scripts directory and run from Scripts>Render>Farmerjoe Submit Render or load farmerjoe_submit.py into blender as a text file and use ALT+P to run it. You can find farmerjoe_submit.py in the bin directory.

Once you have renders submitted you might want to check on their progress, you can telnet to the master server using the same port as in the Farmerjoe.conf file or you can run this command Farmerjoe.xxx --appserver which will run a web server that will allow you to see jobs progress and delete jobs and reset tasks as well as see the slaves connected. To connect to Farmerjoe with a web browser use the URL http://<master server ip/host>:<master server port>

When the render is complete you can retrieve the frames from the jobs directory look for a folder with the name of the job you submitted, once you have copied the frames to a safe place you can use the web gui to delete the job which will also remove the job directory - be warned there is no confirmation it just blows it away.

TODO ^

Bucket renders always render as TARGA files as I am cheating when compositing them, rather than cutting out the rect from the full image I want I simply composite them and rely on the alpha chanel to do the work. I plan on doing this better in the future - possibly using ideasmans composite function fron his blenderfarm script :) and use blender to composite the image parts. The imagemagick's composite also seems to give different results on windows than on linux, the image is still composited but the background colour differs.

There are many small modifications I'd like to add, an auto refresh for the web interface, easier to read progress, more checking etc, however it works pretty well for what I need it for and hopefully others will find it useful to as is.

Id also like to get rid of all the command windows, maybe just have one that runs the other required processes or somthing - suggestions are welcome.

Help ^

If you need help to get Farmerjoe farming email me <lobo.nz (at) gmail (dot) com>
or
go to Freenode IRC Servers in the #blenderchat or #blender channel I am there quite often.

Using Perl instead of Binaries ^

If you want to modify and/or use the perl file rather than use the binaries you will need these extra perl modules

YAML::Tiny
Compress::Zlib

The rest should come standard with perl.

You also need to change these three lines in the farmerjoe_submit.py in the #variables Section if you want the slaves to use the Farmerjoe.pl script too

'linux_farmerjoe'  :'Farmerjoe.linux',
'windows_farmerjoe'  :'Farmerjoe.exe',
'osx_farmerjoe'  :'Farmerjoe.osx'

So that they refer to the Farmerjoe.pl

'linux_farmerjoe'  :'Farmerjoe.pl',
'windows_farmerjoe'  :'Farmerjoe.pl',
'osx_farmerjoe'  :'Farmerjoe.pl'

If you make changes or improvements let me know and please email me a copy so I can include it for others. Also if you have suggestions or trouble please email me - I will try to reply to all emails :) <lobo.nz (at) gmail (dot) com>

Thanks ^

Thanks to ideasman I have used a small portion of his blenderfarm script which I ported to perl to save writing the rect calculations for bucket rendering from scratch. Also thanks to those who have given feedback on Blenderartists, Email and IRC

License ^

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

DISCLAIMER ^

I am paranoid and poor so read this :)

THE PROGRAM IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, BUT WITHOUT ANY WARRANTY. IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. IN NO EVENT WILL THE AUTHOR WILL BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.