running BURP on a network

Message boards : Server backend and mirrors : running BURP on a network
Message board moderation

To post messages, you must log in.

AuthorMessage
tom sparks

Send message
Joined: 16 Jun 06
Posts: 45
Credit: 2,640
RAC: 0
Message 7035 - Posted: 22 Nov 2007, 1:43:03 UTC

Can Burp be run on a community network?

ID: 7035 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7044 - Posted: 25 Nov 2007, 11:29:13 UTC - in response to Message 7035.  

Can Burp be run on a community network?

Could you explain a bit in-depth about what you mean by \"be run\"?
ID: 7044 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
tom sparks

Send message
Joined: 16 Jun 06
Posts: 45
Credit: 2,640
RAC: 0
Message 7056 - Posted: 27 Nov 2007, 8:10:28 UTC

I am looking at setting up a rendering farm
I dont want to set up a rendering farm in-house for community use (when there are computer being under used)
ID: 7056 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Happy Hippo

Send message
Joined: 16 Jul 07
Posts: 15
Credit: 3,223
RAC: 0
Message 7057 - Posted: 27 Nov 2007, 9:30:16 UTC - in response to Message 7056.  

I am looking at setting up a rendering farm
I dont want to set up a rendering farm in-house for community use (when there are computer being under used)


Hm, if I am understanding correctly, you want to use BURP in your local Network to let Blender do distributed rendering...?

I guess there are better solutions than that.

Another possibility is that you have a renderfarm at home but are worried, that BURP could cut your ressources when you want to render something yourself on the farm.
ID: 7057 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7060 - Posted: 27 Nov 2007, 11:17:59 UTC - in response to Message 7057.  
Last modified: 27 Nov 2007, 11:19:17 UTC

Hm, if I am understanding correctly, you want to use BURP in your local Network to let Blender do distributed rendering...?

If this is what you ment - ie. setting up a local renderfarm using BURP but disallowing external access to it - then HH is right, unless your network is very big (more than just 100 machines) there are solutions that were developed with this scenario in mind. One that I\'ve heard mentioned multiple times is DrQueue.

It is true that BURP would do equally well if set up correctly on your network - however, BURP requires a full server installation including webserver(s), data server(s), database server(s) and a mail server. This may be cutting it slightly too sharp if all you want is to manage a few locally controlled machines.

Ok, that said, if you are an organization with plans to utilize more than 100 machines to do distributed rendering and some of these machines are not controlled locally then BURP is the better choice. With its built-in support for redundancy checks, rescheduling and easy-to-use interface it makes things easy for you to run. However, keep in mind that some parts of the BURP server are not yet opensourced. This is something we can talk about.
ID: 7060 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
tom sparks

Send message
Joined: 16 Jun 06
Posts: 45
Credit: 2,640
RAC: 0
Message 7062 - Posted: 27 Nov 2007, 15:15:43 UTC

[quote]
BURP requires a full server installation including webserver(s), data server(s), database server(s) and a mail server. [/quoted]
I knew that, we are going to have a web server on the network for information reasons

[quote]
some of these machines are not controlled locally then BURP is the better choice.
[/quoted]
all the computer well not be controllable

DrQueue is too much work for joe blow (install DrQueue, install blender, etc)
boinc and BURP fits the bill

I currently do not know how many computer well be used for the rendering farm
ID: 7062 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7064 - Posted: 28 Nov 2007, 9:39:07 UTC - in response to Message 7062.  

I currently do not know how many computer well be used for the rendering farm

This is a very important system design parameter. If we are talking about 10000 machines you are going to need different hardware than if it is only 200 or 2.
How many computers are currently connected to the network?

all the computer well not be controllable [...] DrQueue is too much work for joe blow (install DrQueue, install blender, etc)
boinc and BURP fits the bill

What kind of people are on the network? What kind of network are we talking about? Is this a school?
What is your role in this network? Are you a system administrator? What kind of networking and system administration experience do you have?

Can Burp be run on a community network?

I guess the answer to this is: yes it can.
However, as stated in my first post some of the server code is not opensourced. The original plan was to opensource it when it was ready for general public use. However I may have to reschedule this to an earlier date (january/february 08) for several reasons.
When did you have in mind to set up this renderfarm?
ID: 7064 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mip2

Send message
Joined: 12 Jan 07
Posts: 26
Credit: 25,862
RAC: 0
Message 7066 - Posted: 28 Nov 2007, 10:25:34 UTC

Have a look to farmerjoe, it is a easy to use and configure network
render tool.
Nothing to install, only copy to the workstations and run.

http://blender.formworks.co.nz/?p=1

Greetings mib
ID: 7066 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
tom sparks

Send message
Joined: 16 Jun 06
Posts: 45
Credit: 2,640
RAC: 0
Message 7069 - Posted: 29 Nov 2007, 8:46:02 UTC

[quote]I currently do not know how many computer well be used for the rendering farm


This is a very important system design parameter. If we are talking about 10000 machines you are going to need different hardware than if it is only 200 or 2.
How many computers are currently connected to the network?
[/quoted]
the network is not even going yet


[quote]
all the computer well not be controllable [...] DrQueue is too much work for joe blow (install DrQueue, install blender, etc)
boinc and BURP fits the bill


What kind of people are on the network? What kind of network are we talking about? Is this a school?
[/quoted]
the network is going to be mainly a information network
using wireless mesh network (http://en.wikipedia.org/wiki/Wireless_mesh_network)
setup

[quote]
What is your role in this network? Are you a system administrator? What kind of networking and system administration experience do you have?
[/quoted]
I am hoping I only have to look after my region of the town
I have been running a home network, I was going to set up a lan gaming network (never got going :( third-party miss-communication)
[quote]
Can Burp be run on a community network?


I guess the answer to this is: yes it can.
However, as stated in my first post some of the server code is not opensourced. The original plan was to opensource it when it was ready for general public use. However I may have to reschedule this to an earlier date (january/february 08) for several reasons.
When did you have in mind to set up this renderfarm?
[/quoted]
one of the town plan for the youth is to show them how to do 3d modeling
short films, etc

the other reason I am looking at boinc is the town has unknown amount of footage
and very few if any people have seen the footage
so I am looking at youtube/video blog like servce and boinc well be the video conversion backend
ID: 7069 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
tom sparks

Send message
Joined: 16 Jun 06
Posts: 45
Credit: 2,640
RAC: 0
Message 7071 - Posted: 29 Nov 2007, 9:07:18 UTC

I am looking use the gOS dev kits motherboards http://www.clubit.com/product_detail.cfm?itemno=A4842001 all i need to add is ram, hard drive and PSU
ID: 7071 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
noderaser
Project donor
Avatar

Send message
Joined: 28 Mar 06
Posts: 516
Credit: 1,567,702
RAC: 0
Message 7074 - Posted: 29 Nov 2007, 20:59:29 UTC

I don\'t think that BOINC would be a good option for converting video, due to the extreme bandwidth and storage requirements that would be required of the server, and each client computer.
Click here to see My Detailed BOINC Stats
ID: 7074 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7085 - Posted: 2 Dec 2007, 18:40:24 UTC

Tom, I suggest you try installing a regular BOINC server and run a few test workunits through it first. The BURP server code is not going opensource in december (simply too much work to get it worthy for that).
ID: 7085 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Raphael Nadeau

Send message
Joined: 8 Mar 11
Posts: 6
Credit: 58,784
RAC: 0
Message 10834 - Posted: 8 Mar 2011, 19:42:44 UTC - in response to Message 7085.  

Hello,

I know I'm bringing back an old subject, but I am very much interested in the possibility to run a "community" BURP project.

I would like to apologize in advance if I misunderstood the purpose or if my question is misplaced. If you point me to the appropriate manual, I will read it.

That being said, my friend is doing some very heavy Blender"ing" and needs a farm in order to render the whole project. We figured that with his computer alone he would need somewhere between 3 to 6 month.

Because of budget, he cannot build a real farm. Although I have tried a project with loki (http://loki-render.berlios.de/), it does work well as long as my computer is on the same network...

There is a lot of people, family and friend that would like to help them, but we cannot let our computer at his place. If I could make a BURP server, I could setup everything so that all of us, with our personal computer, could give him a little help with his project.

I do not want to be mean, but it is not that we do not want to help the BURP open community, it is just that as of right now, their project is urgent and we want to prioritize on them first. If it is possible to do with (and make sure the project are automatically accepted by the community) I would organize everything so we will all join the community instead of a private BURP server.

So if it is possible, I would VERY MUCH APPRECIATE the help and/or any info on how to do this.

Personnaly, I think your initiative is great and I plan to contribute in the future, even if I never did any 3D art/render before.

Best Regards
Raph!
ID: 10834 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 10835 - Posted: 8 Mar 2011, 21:55:21 UTC - in response to Message 10834.  
Last modified: 8 Mar 2011, 22:03:58 UTC

[...] I am very much interested in the possibility to run a "community" BURP project. [...] my friend is doing some very heavy Blender"ing" [...] We figured that with his computer alone he would need somewhere between 3 to 6 month.
[...]
There is a lot of people, family and friend that would like to help them, but we cannot let our computer at his place. If I could make a BURP server, I could setup everything so that all of us, with our personal computer, could give him a little help with his project.

It is theoretically possible to do so since BURP is now entirely open source; Renderfarm.fi did it and I'm aware of a few other installations on smaller networks.
You should, however, be aware that the technical skills currently required for setting up a BURP farm means that you will need to be very experienced with Linux and servers.

I do not want to be mean, but it is not that we do not want to help the BURP open community, it is just that as of right now, their project is urgent and we want to prioritize on them first. If it is possible to do with (and make sure the project are automatically accepted by the community) I would organize everything so we will all join the community instead of a private BURP server.

That is a very interesting idea: using the BURP site to create smaller "closed" communities with their own site admins, their own sessions and their own rendering nodes.
Currently the system can not handle this kind of scenario - and it would take some time to implement.

What are the specs on your animation? (file size, memory requirements and so on)
Right now the portion of the farm with less than 6GB of RAM is pretty much idle. Keep in mind, though, that the farm is currently running Blender 2.4x. Also check out Renderfarm.fi who have a quicker submit-to-render time and are running Blender 2.5x on their BURP farm. If you are ok with having the output under an open license then using online farms should get you pretty far.

Alternatively, let me suggest that you VPN or tunnel machines (or the controller) onto the local network and use a standard render manager. Oh, and these solutions are not mutually exclusive.

I can't really give your any better advice without knowing the specifics.
ID: 10835 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Raphael Nadeau

Send message
Joined: 8 Mar 11
Posts: 6
Credit: 58,784
RAC: 0
Message 10837 - Posted: 8 Mar 2011, 23:16:50 UTC - in response to Message 10835.  

Thank you very much for your quick and precise answer.

Like you mentioned, one of the problems is that the whole project is closed source (for at least until January 2012) by contract. After, it will depend if the artist is willing to open it up. (I am not the artist, I am the big brother Linux geek who is trying to help)

I will look into VPN (I had good succes with loki) and into Renderfarm.fi, but I would like to know where I can get access to the source code. I just went to sourceforge.net and there are no files available. Do you know where I can get them?

As for the specs of the animation, I do not know yet. If I remember correctly, the whole project is about 500 MB, but I have to do some benchmark to verify how much RAM is needed. I know that once rendered, it should be 40 minutes of 1080p at 24 FPS.

The "artist" in question is using a Mac with two quad Core i7 with 48 GB of RAM, so he never really bothered to check how much he was using.

Again, thank you for this precious information
Raph!
ID: 10837 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 10838 - Posted: 9 Mar 2011, 7:46:32 UTC - in response to Message 10837.  

I would like to know where I can get access to the source code.

At the bottom of the main menu there is a link with "Help us code". Check the installation guide in the root of the main source tree for more info.
If you run into any trouble you can get some help by posting to the Problems and Help forum.
ID: 10838 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Raphael Nadeau

Send message
Joined: 8 Mar 11
Posts: 6
Credit: 58,784
RAC: 0
Message 10839 - Posted: 9 Mar 2011, 13:00:15 UTC - in response to Message 10838.  

Thanks
ID: 10839 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Server backend and mirrors : running BURP on a network