No Checkpointing??


Advanced search

Message boards : Number crunching : No Checkpointing??

Author Message
keputnam
Send message
Joined: 31 Mar 05
Posts: 15
Credit: 109,233
RAC: 130
Message 13436 - Posted: 3 Dec 2014, 9:24:04 UTC

We just took a power hit, and a Blender job with over 8 hours of wall clock time , came back up with 00:00:00 ??
____________

baracutio
Project donor
Send message
Joined: 29 Mar 05
Posts: 96
Credit: 174,604
RAC: 0
Message 13437 - Posted: 3 Dec 2014, 15:51:40 UTC

that's right. there are no checkpoints!

Profile noderaser
Project donor
Avatar
Send message
Joined: 28 Mar 06
Posts: 507
Credit: 1,551,072
RAC: 201
Message 13443 - Posted: 4 Dec 2014, 3:05:05 UTC

A limitation of Blender.
____________

keputnam
Send message
Joined: 31 Mar 05
Posts: 15
Credit: 109,233
RAC: 130
Message 13447 - Posted: 4 Dec 2014, 9:27:26 UTC - in response to Message 13443.

A limitation of Blender.


That is utterly absurd!

____________

Profile noderaser
Project donor
Avatar
Send message
Joined: 28 Mar 06
Posts: 507
Credit: 1,551,072
RAC: 201
Message 13450 - Posted: 5 Dec 2014, 3:14:55 UTC

http://www.blender.org/get-involved/
____________

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 13452 - Posted: 5 Dec 2014, 16:57:52 UTC
Last modified: 5 Dec 2014, 16:59:06 UTC

Rather than using checkpointing we try to split frames into smaller chunks - typically of around 1 hour sizes. Unfortunately not all animations allow this - some must be rendered in full frames because of certain special effects. There are ways to get around this but most artists prefer the full-frame approach because it is really easy to understand and works exactly like normal rendering works.

As a matter of fact some work has gone into thinking up different ways to checkpoint Blender without direct support in Blender itself (which is a quite complicated thing) - stuff like storing a complete memory dump of the program for example. Unfortunately (or security wise, fortunately, actually!) most new operating systems do not allow this kind of direct memory manipulation and will instead load programs and require programs to be loaded differently every time they are started (randomized program memory allocation strategies).

A lot of work also went into analyzing Blender's rendering model, even identifying the relevant portions of code that need to be changed. Unfortunately (or maybe again fortunately?) Blender is in a constant flux of new features - some of them being entirely new renderers like Cycles. These new renderers/features do things differently than the old ones, adding complexity and constant variability to a system that would be capable of storing render progress snapshots and restart points. It turns out the task of adding such a system is difficult and not something that you could just do in an afternoon; it would likely require a rewrite of major portions of the entire rendering subsystem.
As with everything opensource: it will happen if someone sits down and actually writes the code to make it happen.

This may, of course, be utterly absurd.

keputnam
Send message
Joined: 31 Mar 05
Posts: 15
Credit: 109,233
RAC: 130
Message 13453 - Posted: 7 Dec 2014, 7:44:29 UTC - in response to Message 13452.
Last modified: 7 Dec 2014, 7:45:05 UTC

Guess I was just unlucky to have two 200K+ second WUs, when I applied service and took a power hit, then
____________

AEM74
Send message
Joined: 1 Dec 15
Posts: 5
Credit: 207,905
RAC: 0
Message 14254 - Posted: 3 Jan 2016, 23:53:39 UTC

If I snooze BOINC, does it start all over again too?

Profile noderaser
Project donor
Avatar
Send message
Joined: 28 Mar 06
Posts: 507
Credit: 1,551,072
RAC: 201
Message 14255 - Posted: 4 Jan 2016, 0:37:31 UTC

No, snoozing will just pause computation, and will pick up where it left off when you resume.
____________

AEM74
Send message
Joined: 1 Dec 15
Posts: 5
Credit: 207,905
RAC: 0
Message 14258 - Posted: 4 Jan 2016, 18:13:34 UTC - in response to Message 14255.

Should have asked earlier so I didn't have to abort all those tasks. :(

nanoprobe
Send message
Joined: 28 Nov 14
Posts: 1
Credit: 54,906
RAC: 0
Message 14260 - Posted: 6 Jan 2016, 1:50:56 UTC - in response to Message 13453.

Guess I was just unlucky to have two 200K+ second WUs, when I applied service and took a power hit, then

A UPS would usually take care of any short power interruptions. Saves losing work and wear and tear on your equipment.


Post to thread

Message boards : Number crunching : No Checkpointing??