Blender 2.70 to hit the farm soon


Advanced search

Message boards : Client : Blender 2.70 to hit the farm soon

1 · 2 · Next
Author Message
Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12489 - Posted: 6 Apr 2014, 17:54:07 UTC
Last modified: 6 Apr 2014, 17:59:28 UTC

As soon as we have cleared the queue of currently rendering sessions a new client will be sent to the farm. It contains


  • an upgrade to Blender 2.70
  • better compatibility for Linux out of the box
  • a way to switch to native libs on Linux platforms if the out-of-the-box solution doesn't work
  • more compact logging
  • a new thread and process control model for all platforms
  • updated Cycles time estimate code, still extremely experimental
  • preliminary support for handling Blender features that require temporary disk space ("Cache BVH", "Save Buffers", "Persisting Images" etc.)
  • a merge of features from Sunflower, including multithreading support



A lot of stuff has changed, so expect it to be hellishly unstable the first few weeks.

funkydude
Send message
Joined: 23 Dec 13
Posts: 275
Credit: 2,478,281
RAC: 0
Message 12492 - Posted: 6 Apr 2014, 23:13:09 UTC

Wow, this seems like a seriously major update.

Better Linux support... thank you, thank you, thank you, thank you, thank you, thank you! Look forward to testing this on my laptop. 1 more step towards moving to Linux on the desktop if it works.

Threading support... will we only be allocated 1 task now instead of 1 per core? I can see that being very beneficial for tasks with long durations.

Can't tasks that require temp space just use RAM?

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12495 - Posted: 7 Apr 2014, 17:21:45 UTC - in response to Message 12492.
Last modified: 7 Apr 2014, 17:44:09 UTC

Wow, this seems like a seriously major update.

It's pretty much 1 year of tiny patches rather than one big change. That counts too, of course.

Threading support... will we only be allocated 1 task now instead of 1 per core?

Yes, in the beginning. We're going to experiment a bit with different setups. Neither 1 task per core nor 1 task per machine with all cores are optimal. We will try out different things in between that.

Can't tasks that require temp space just use RAM?

Some of them, sure, others not so much. Sunflower sessions made use of this feature to store around 20GB of intermediate data on disk on top of a 5-10GB RAM requirement. The typical use case is to use this for one of the best anti-aliasing methods in Blender but it is really eating up RAM if you don't swap it to disk.

Achim
Send message
Joined: 17 May 05
Posts: 182
Credit: 2,527,860
RAC: 1,092
Message 12536 - Posted: 11 Apr 2014, 5:59:59 UTC

All my (linux) tasks failed.
E.g. http://burp.renderfarming.net/result.php?resultid=7820322

Looks like the rendering works, but afterwards I always get messages like this:
11-Apr-2014 06:18:35 [BURP] Computation for task ses0000002038frm0000000295prt00001_2 finished
11-Apr-2014 06:18:35 [BURP] Output file ses0000002038frm0000000295prt00001_2_0 for task ses0000002038frm0000000295prt00001_2 absent

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12537 - Posted: 11 Apr 2014, 17:08:59 UTC
Last modified: 11 Apr 2014, 17:09:41 UTC

Well then you get to be the first one to try out the new "native mode" for Linux. Create a file called "native" in the project directory (that's /var/lib/boinc-client/projects/burp.renderfarming.net for you) and see if anything changes in the output.
Before you do this you should make sure that Blender 2.70 x86_64 from http://blender.org works on your machine.

Achim
Send message
Joined: 17 May 05
Posts: 182
Credit: 2,527,860
RAC: 1,092
Message 12542 - Posted: 11 Apr 2014, 20:04:07 UTC

Still the same.
http://burp.renderfarming.net/result.php?resultid=7820450

I don't see anything hint concerning the "native" file in the output.
But maybe you know where to search.

Unfortunatly I have to switch off the computer for a while, and check back, when it is switched on again


Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12543 - Posted: 11 Apr 2014, 20:41:05 UTC - in response to Message 12542.
Last modified: 27 Apr 2014, 9:39:28 UTC

Interesting. I'm looking at the code and there may be a missing slash in the path for the file that you have to create. So in other words it may very well be that the file has to be "/var/lib/boinc-client/projects/burp.renderfarming.netnative" rather than "/var/lib/boinc-client/projects/burp.renderfarming.net/native".
The result should be that it prints out "Skipping preloader" rather than "CHMODing preloader" when it starts.

Note: In the newest client this issue was fixed and the name is now correctly "native" in the project folder.

Unfortunatly I have to switch off the computer for a while, and check back, when it is switched on again

No problem at all

funkydude
Send message
Joined: 23 Dec 13
Posts: 275
Credit: 2,478,281
RAC: 0
Message 12544 - Posted: 12 Apr 2014, 11:18:35 UTC - in response to Message 12543.

Still haven't tried it myself but the returned results seem to indicate there's a lot of errors for Linux users :/

On another note Blender 2.70a was released. No change log but I'm guess it's bugfixes we should grab?

Duff
Send message
Joined: 7 Apr 14
Posts: 1
Credit: 1,481,368
RAC: 0
Message 12546 - Posted: 12 Apr 2014, 16:05:16 UTC

Just want to say that it's working here, on a gentoo system running a 3.12 series kernel:

http://burp.renderfarming.net/result.php?resultid=7821372

Once I touched the file (/var/lib/boinc/projects/burp.renderfarming.netnative in my case) and restarted the daemon, it started crunching away with 8 cores. No more glibc missing symbol related errors, and no more running win7 in a VirtualBox required for me.

My main worry now is waiting for my new noctua heatsink to show up so I can replace the inadequate 212 evo that's currently installed.

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12547 - Posted: 12 Apr 2014, 16:51:13 UTC - in response to Message 12546.

Really nice to hear that it worked. The path to the "native" file will be fixed in the next release so you may want to create both for now.

funkydude
Send message
Joined: 23 Dec 13
Posts: 275
Credit: 2,478,281
RAC: 0
Message 12553 - Posted: 13 Apr 2014, 11:43:36 UTC - in response to Message 12547.

If the native method works and the current one doesn't, shouldn't that be the default? Also can you explain what you mean by native method? In my mind it means using the libraries already on your machine.

Will the next release update to 2.7a?

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12556 - Posted: 13 Apr 2014, 19:17:17 UTC
Last modified: 13 Apr 2014, 19:27:15 UTC

Your understanding of "native" in this context is correct, that is exactly what it does.

It isn't black or white, though. The current (old?) delivery method contains all the resources (think libraries, environment etc.) to start Blender even on systems that do not have any. It is one of three ways to launch a program.
The second method is to trust the system to have all required libraries installed natively through the system's own package manager, this is what the "native" file does. It is a bit more involved as it requires whoever owns the machine to perform some tasks in order to get it to work.
The third method, which is still in the works but has been tested here before, is a kind of combination of the two: using part of the system's own environment to load the libraries contained in the .zip. In rough numbers method 1 should take care of the most popular setups which hopefully means more than 50%, method 3 an additional 25% and method 2 is the final fallback which together with the anonymous platform brings it to around 100%.
Historically the two automatic methods are fairly reliable in the way that if they work they will usually return correct results. "Native" has the drawback that it may seem to work but can return incorrect results in certain situations.

What you may be seeing right now is a few Linux boxes trashing through hundreds of workunits per minute and failing instantly because their failure counters were reset recently.

funkydude
Send message
Joined: 23 Dec 13
Posts: 275
Credit: 2,478,281
RAC: 0
Message 12558 - Posted: 13 Apr 2014, 23:41:28 UTC - in response to Message 12556.

Thanks for explaining. I guess what I don't understand is why method 1 is even failing at all? Permissions?

Method 2 could be better achieved for distros like Debian/Ubuntu by having a BURP PPA with the required libraries. Though I guess that still wouldn't assure you that a specific version of a library will be used. I'm assuming the issues with method 2 are due to differing versions of libraries on host machines.

Achim
Send message
Joined: 17 May 05
Posts: 182
Credit: 2,527,860
RAC: 1,092
Message 12616 - Posted: 25 Apr 2014, 6:02:14 UTC

Don't know why, but since the new client and the restart of the maschine it works :-)
Now I see as well something in the output.

Janus do you get the info in your statistics, how many linux maschine use the native mode , and the other way around?

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12620 - Posted: 26 Apr 2014, 7:52:09 UTC - in response to Message 12616.

Great!

And no, there's no stats yet. Most machines are using the default setting, though (from a quick look in the DB).

JZD
Send message
Joined: 30 Dec 11
Posts: 89
Credit: 3,136,413
RAC: 5,626
Message 12628 - Posted: 26 Apr 2014, 20:53:02 UTC - in response to Message 12620.

Hi Janus, can you please go back version of Blender v4.86 (mt) in native mode? Here it correctly. Blender Version v4.87 (mt) is in my error.

funkydude
Send message
Joined: 23 Dec 13
Posts: 275
Credit: 2,478,281
RAC: 0
Message 12629 - Posted: 26 Apr 2014, 20:54:34 UTC - in response to Message 12620.

Just wiped and updated one of my laptops to 14.04 and decided I'd try BURP. Unfortunately, it failed.

http://burp.renderfarming.net/result.php?resultid=7839458

So now I'm going to try native mode, the question is, why did it fail? It's a pretty vanilla install, is something missing?

funkydude
Send message
Joined: 23 Dec 13
Posts: 275
Credit: 2,478,281
RAC: 0
Message 12630 - Posted: 26 Apr 2014, 21:29:38 UTC - in response to Message 12629.

Native worked... I don't understand why. I don't have Blender installed and I assume I also don't have the required libs installed unless they exist in Ubuntu vanilla...

http://burp.renderfarming.net/result.php?resultid=7839553

Should I not have Blender installed to make sure the required libs are installed?

Achim
Send message
Joined: 17 May 05
Posts: 182
Credit: 2,527,860
RAC: 1,092
Message 12632 - Posted: 26 Apr 2014, 22:01:31 UTC

Hi Janus,

maybe you should update this message

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12634 - Posted: 27 Apr 2014, 9:43:57 UTC

JZD: Check the name, you are not using native mode with the new client. The name of the trigger file was fixed and now it is correctly named "native" and should be located inside the burp.renderfarming.net dir together with all the other BURP files.

Funkydude: If you are interested you can check out the libraries used by unpacking the blender?.zip in the burp.renderfarming.net project directory somewhere and running:
ldd ./blender
It will list the libraries used and where it found your native ones (if it did).

1 · 2 · Next
Post to thread

Message boards : Client : Blender 2.70 to hit the farm soon