Blender python scene checking script

Message boards : Client : Blender python scene checking script
Message board moderation

To post messages, you must log in.

AuthorMessage
AC
Project donor
Avatar

Send message
Joined: 30 Sep 07
Posts: 121
Credit: 143,874
RAC: 0
Message 7788 - Posted: 19 Feb 2008, 9:31:54 UTC

Well I\'d offer to help, but I don\'t know what I\'d be able to do. I went to a school that\'s known for its database and networking classes, and managed to avoid those and specialize in graphics instead. End result: jack of all trades, master of none.

I see \"parameter checking\" on the list -- does that mean checking the .blend file for problematic settings, or checking the BURP session settings that the user entered? A while ago I toyed with making a blender script to check for some of the obvious problems (e.g. multipart SSS) but I never finished it. If it would be helpful, I could work on that some more...
ID: 7788 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7796 - Posted: 20 Feb 2008, 7:28:26 UTC - in response to Message 7788.  
Last modified: 20 Feb 2008, 7:34:02 UTC

I see \"parameter checking\" on the list -- does that mean checking the .blend file for problematic settings, or checking the BURP session settings that the user entered?

The first one mainly. Checking of the settings as well as fetching out stuff like resolution etc. for the upload guide is done by a patched Blender executable on the server rather than by a python script.

A while ago I toyed with making a blender script to check for some of the obvious problems (e.g. multipart SSS) but I never finished it. If it would be helpful, I could work on that some more...

That would indeed be very helpful as it would also serve as a central part of a future upload script. If you are interested in working on this here is a list of things to check (there\'s potentially more than these, these are off the top of my head):
- All file access must be either to something packed in the Blend file or relative paths. In case of relative paths it must not include the \"up-one-directory\" (aka. ../) token or similar ways of going up the directory path. (\"campbel\" talks a bit about it here.)
- All file access must be to files that exist (with generated files being an exception)
- SSS cannot be used with multipart rendering (we may start using the split setting from Blender to initialize the one on the website)
- Rendering a single frame without any split is better to do locally
- Physics should have been baked
- Panorama rendering is not allowed due to technical limitations
- Border rendering is not allowed (because it messes up with the border rendering instructions from the server)
Current limitations:
- Physics must have been baked (no serverside baking yet)
- No linking to other files is allowed, only files inside the .blend (no library calls allowed yet)
- No linking to AVI files, even in the .blend
Current bugs causing workarounds:
- Using the sequencer disallows splitframe rendering and all subscenes should have a split of 100x100
- No support for EXR, use PNG
- No support for save-buffers-to-disk while rendering, simply disable it
Current optimization tricks:
- Textures should be in a compressed format rather than in a raw image format
- If the scenefile contains python scripts that were used during the making of the scene these should be stripped out if they aren\'t necessary for the rendering of the scene

If you have the time it would truely be interesting to have a set of python functions that could check each of these. For every minute of work you do on this, even if you only do part of it, you will be freeing up time for either me or one of the other devs.
ID: 7796 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
AC
Project donor
Avatar

Send message
Joined: 30 Sep 07
Posts: 121
Credit: 143,874
RAC: 0
Message 7805 - Posted: 21 Feb 2008, 3:02:07 UTC - in response to Message 7796.  
Last modified: 21 Feb 2008, 3:35:14 UTC

How does (or will) BURP handle .blend files containing multiple scenes? It may make a difference in some of these functions. For example, multipart rendering is a per-scene setting.
ID: 7805 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7806 - Posted: 21 Feb 2008, 6:33:30 UTC - in response to Message 7805.  

How does (or will) BURP handle .blend files containing multiple scenes? It may make a difference in some of these functions. For example, multipart rendering is a per-scene setting.

It uses \"the active scene\". The active scene is also the scene that will be displayed when you load the .blend into Blender. The user should be careful about setting the correct active scene.

For instance in the case where the sequencer is used to combine a number of scenes it is important that the .blend is saved with the sequencer scene as the active scene.
ID: 7806 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
AC
Project donor
Avatar

Send message
Joined: 30 Sep 07
Posts: 121
Credit: 143,874
RAC: 0
Message 7814 - Posted: 22 Feb 2008, 10:24:23 UTC
Last modified: 22 Feb 2008, 10:24:52 UTC

I\'m working on the texture checks, possible values for texture.type are
STILL: Allowed.
MOVIE: Not allowed by BURP.
SEQUENCE: Are image sequences going to be allowed?
GENERATED: I assume these are allowed?
ID: 7814 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7816 - Posted: 22 Feb 2008, 12:29:07 UTC - in response to Message 7814.  

I\'m working on the texture checks, possible values for texture.type are
STILL: Allowed.
MOVIE: Not allowed by BURP.
SEQUENCE: Are image sequences going to be allowed?
GENERATED: I assume these are allowed?

Good question. I think you may have to actually test it to find out. If the sequence can be baked into the .blend file then I guess it\'s ok.
All generated textures should be generated into the current directory if they use file access (no access outside of the dir).
ID: 7816 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
AC
Project donor
Avatar

Send message
Joined: 30 Sep 07
Posts: 121
Credit: 143,874
RAC: 0
Message 7832 - Posted: 24 Feb 2008, 19:07:24 UTC - in response to Message 7796.  

Textures should be in a compressed format rather than in a raw image format


Does this refer to the file format (e.g., PNG) of the input images? I don\'t know of any built-in way to check that, other than just looking at the extension on the filename and comparing it to a blacklist. It\'s not a great solution... any thoughts/suggestions?

Blender apparently uses QuickTime to read the following formats, and I\'m assuming they will probably all be blacklisted: gif, tif, tiff, pct, pict, pntg, psd, qtif

I think that all of Blender\'s internally supported file formats are capable of at least RLE compression, but some of them should probably be on the list. It supports: bmp, bw, iff, jpeg, jpg, lbm, png, rgb, sgi, tga

If this approach makes sense, then which of the interal formats should be blacklisted?

Would it be easier and/or safer to have a whitelist?

What should be done for extensionless files -- are they assumed good or assumed bad?
ID: 7832 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7833 - Posted: 24 Feb 2008, 20:11:01 UTC - in response to Message 7832.  
Last modified: 24 Feb 2008, 20:14:30 UTC

I guess the best thing is to have a whitelist of formats and a blacklist.
What we want to catch is the people who didn\'t consider compression at all (the blacklist). For stuff that is compressed but not the best file format we should just produce a warning or a notice (not on whitelist and not blacklisted either).

gif, png, jpg are among the good formats (for different reasons).
bmp and similar non-compressed formats are always bad.

Extensionless files can be considered to be the reason for a warning/notice as well.
ID: 7833 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
AC
Project donor
Avatar

Send message
Joined: 30 Sep 07
Posts: 121
Credit: 143,874
RAC: 0
Message 7842 - Posted: 26 Feb 2008, 4:43:08 UTC - in response to Message 7796.  

- No linking to AVI files, even in the .blend
How about mpeg, mov, etc.?
ID: 7842 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7850 - Posted: 27 Feb 2008, 17:15:35 UTC - in response to Message 7842.  

- No linking to AVI files, even in the .blend
How about mpeg, mov, etc.?

Same problem, I expect - although this has not been confirmed.
The thing is that AVI files require unpacking before they are accessible. You can test support in the following way:
1) Create a scene with something that has a video file as a texture.
2) Pack the videofile with the .blend and close Blender
3) Move the original (nonpacked) videofile somewhere else (or rename it so that Blender cannot reach it)
4) Start Blender again and hit render without unpacking the .blend file to local storage.
If the scene renders with the video on the texture it is fine, if not the format is not supported yet.

Ordinary (image) textures can be read by Blender without unpacking them from the .blend file first, so they are supported.

At some point BURP will support .blend unpacking or attaching files in a similar fashion, it just doesn\'t support it yet due to the many security-related issues.
ID: 7850 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
AC
Project donor
Avatar

Send message
Joined: 30 Sep 07
Posts: 121
Credit: 143,874
RAC: 0
Message 7855 - Posted: 28 Feb 2008, 18:41:30 UTC - in response to Message 7850.  

Well, I can confirm that MOVs have to be unpacked. I haven\'t yet found an mpeg that Blender wants to read, but I\'m just going to assume it\'s the same as avi & mov.

It also appears that Blender packs only one image from a sequence (either first or current, I\'m not sure which). It reads that one image fine without unpacking, but unpacking only restores that one image, and not the whole sequence.

This\'ll probably [hopefully!] all be changed when 2.46 comes out in a month or two ;-)
ID: 7855 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7926 - Posted: 20 Mar 2008, 8:51:30 UTC - in response to Message 7855.  
Last modified: 20 Mar 2008, 8:52:01 UTC

It also appears that Blender packs only one image from a sequence (either first or current, I\'m not sure which). It reads that one image fine without unpacking, but unpacking only restores that one image, and not the whole sequence.

Hm... maybe a feature suggestion at the Blender site would be in place here. I don\'t know how the current SVN version does this, though - maybe it already has been fixed.

A month ago \"Campbell\" posted (in a comment) some scripts they used to check up the Peach session files before shipping them to the SUN renderfarm. It may be worth checking that out. A lot of the limitations they face are the exact same that BURP users will face.
ID: 7926 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
AC
Project donor
Avatar

Send message
Joined: 30 Sep 07
Posts: 121
Credit: 143,874
RAC: 0
Message 7928 - Posted: 20 Mar 2008, 9:20:37 UTC - in response to Message 7926.  
Last modified: 20 Mar 2008, 9:22:41 UTC

I will look into it -- I have gotten bogged down with a potential new job (everybody cross your fingers!) and haven\'t made much progress on this lately, but I haven\'t abandoned the scripts and am still working on them.

I\'m also hoping to submit a scene sometime soon, and I\'m hoping I\'ll be able to check it with the script before I send it! ;-)
ID: 7928 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7930 - Posted: 20 Mar 2008, 15:39:41 UTC - in response to Message 7928.  

I will look into it -- I have gotten bogged down with a potential new job (everybody cross your fingers!) and haven\'t made much progress on this lately, but I haven\'t abandoned the scripts and am still working on them.

I\'m also hoping to submit a scene sometime soon, and I\'m hoping I\'ll be able to check it with the script before I send it! ;-)

Hehe, no problem, we are not in any hurry. I whish you the best of luck with the job!
ID: 7930 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Istvan Burbank
Avatar

Send message
Joined: 3 Apr 08
Posts: 312
Credit: 58,920
RAC: 0
Message 9438 - Posted: 19 Apr 2009, 17:07:54 UTC

I am working on a similar project, for checking .blends, and while doing some googling to see if there were any tips I found this!

Are there any versions, ideas, tests etc. that could be posted here? I would offer to share my code with BURP when it is done, but mine is designed to be quite different.

~thanks, Istvan
ID: 9438 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 9443 - Posted: 20 Apr 2009, 18:01:11 UTC

I know that jesterKing is working on a combined testing and uploading script. You may want to keep an eye on this thread: Uploader script coming soon [ORE].
ID: 9443 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Client : Blender python scene checking script