Posts by Janus

1) Message boards : Server backend and mirrors : Extended Monday maintenance, Starting already Saturday 7nth of October 2017 (Message 15263)
Posted 4 days ago by Profile Janus
Inspection of the fans was completed and the CPU heatsink replaced. New fans for storage were ordered and are going to be installed this weekend. Expect about half an hour of downtime later today.
2) Message boards : Server backend and mirrors : Extended Monday maintenance, Starting already Saturday 7nth of October 2017 (Message 15262)
Posted 11 days ago by Profile Janus
Servers will be up and down a bit this weekend in order to service a couple of aging fan ball bearings that sound like they need a bit of attention.
While the main web-server is down it will also have its stock CPU heatsink replaced with an improved version. The stock one has proven insufficient to cool the system under load.
3) Message boards : Comments and discussion : 3396 (Message 15260)
Posted 22 days ago by Profile Janus
Seems so - it looks very noisy. Switching to best-effort mode. Credit will of course be granted even for the units marked as incorrect.

Ironically, artists tend to use as few samples as possible to save time. But when rendering on BURP with Cycles the noise will actually cause the session to become slower because it cannot validate.
The trick is to use deterministic noise or to use more samples (up to a certain limit, of course - there are diminishing returns) to avoid the noise in the first place and then add back noise afterwards if required.

Comment to the author:
This looks awesome! Now we just need a fly-by animation ;)
In the future: Try using a deterministic sampling method on the camera or use more samples.
The resolution you put into the x/y input fields inside Blender is the one that will be used. The % field is not used.
4) Message boards : Server backend and mirrors : Data Compression (Message 15259)
Posted 22 days ago by Profile Janus
The initial tests from the past two weeks showed a number of issues with the compression system and how jobs were being scheduled across the server cluster. Most of the performance issues have been ironed out now.

One interesting issue was that the tests themselves were bugged: When a session is compressed the system carries out a test to see that it can decompress the compressed session and get the original data back. The original and decompressed data is compared using strict validation (a variant of our normal validator) before the original data is disconnected (discarded from active storage). However, it turned out that when a frame contained no colourful pixels it could be compressed with greyscale instead of RGB (saving even more space) but when the test tried to re-open the image it would incorrectly interpret the greyscale image as being different and mark the frame as failed.
It turned out to be related to how Java's built-in image reader maps gamma for different kinds of images. By-passing the gamma correction step and reading the values directly from the raw image not only fixed the issue but also had the interesting side-effect of making our normal validator (the one targeted at Blender's internal renderer) a bit faster too.

This data compression background task has had a few additional resources assigned to it and will continue at a slightly faster pace over the next few weeks. It will be interesting to see if it hits another snag or if it will just keep going.
5) Message boards : Comments and discussion : 1950 (Message 15256)
Posted 23 days ago by Profile Janus
Or some performance changes made to the validator yesterday - definitely taking a closer look at this when it is done. It looks like all the differences are confined to one channel in the output and almost always the last one.

But yeah, it could also be just the render that is noisy in some way.
6) Message boards : Comments and discussion : 1950 (Message 15253)
Posted 23 days ago by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

It is now re-purposed as a client-test of the new 5.12 client (Blender version 2.79) for Linux.
Also, it has a very long scene preparation phase and a very short render phase.
7) Message boards : Comments and discussion : 1965 (Message 15252)
Posted 23 days ago by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

It is now re-purposed as a client-test of the new 5.12 client (Blender version 2.79) for Windows.
Also, there is something odd about the colour format.
8) Message boards : Comments and discussion : 3396 (Message 15251)
Posted 24 days ago by Profile Janus
This upload is waiting for the new 2.79 to be released onto the farm
9) Message boards : Client : Looking for bold MAC users (Message 15246)
Posted 13 Sep 2017 by Profile Janus
Looks like we have the first returned result from a CUDA-enabled MAC. It did actually complete the task and the WU is now waiting for a wingman to comeplete it.
It appears that there are very few MACs with CUDA attached to the project. It took almost a month before anyone requested a single workunit in that category.
10) Message boards : Comments and discussion : 2967 (Message 15245)
Posted 13 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, it has been rejected because the upload is a .ZIP-file and we currently do not support .blend-files inside .zip because .blend-files already have the option to GZIP the data inside them.
11) Message boards : Comments and discussion : 1509 (Message 15243)
Posted 12 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, it has been rejected because the upload is a LightWave scene file and unfortunately the proprietary commercial LightWave renderer is not among the supported rendering engines on this opensource farm.
12) Message boards : Comments and discussion : 1731 (Message 15242)
Posted 12 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, it has been rejected because the upload is a JPEG image file and doesn't describe a 3D scene to be rendered in any of the currently supported renderers.
13) Message boards : Comments and discussion : 1834 (Message 15241)
Posted 12 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, it has been rejected because the upload is what appears to be a file containing 29MB of seemingly unidentifiable random data and doesn't describe a 3D scene to be rendered in any of the currently supported renderers.
14) Message boards : Comments and discussion : 1983 (Message 15240)
Posted 12 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, it has been rejected because the upload is a PNG image file and doesn't describe a 3D scene to be rendered in any of the currently supported renderers.
15) Message boards : Comments and discussion : 2909 (Message 15239)
Posted 12 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, it has been rejected because it is an AVI movie file and doesn't describe a 3D scene to be rendered in any of the currently supported renderers.
16) Message boards : Server backend and mirrors : Data Compression (Message 15238)
Posted 12 Sep 2017 by Profile Janus
Feature, yay! A BARF milestone subproject.

Part of the running cost of BURP is storage. If we are to open up BURP and scale it up with a factor of 10x or 100x in the future then we need to be much more conscious about server-side storage. Currently, for a number of reasons, the strategy has been to store literally everything (including multiple copies of each frame rendered from each render node, both parts and the stitched frames etc.). That way when something goes wrong, it costs very little to recreate it from some deeper storage layer.
In recent years fewer and fewer things have gone wrong where the extra storage was needed, and when the custom Cycles validator project is done there is a very large portion of that kind of "almost but not quite duplicate" storage that can be freed up for other purposes.
In the future the hope is to make the service a lot more slim, so that it scales better without unnecessary waste of storage.

The session data that will be put in long-term storage is going to consist primarily of

  • One canonical copy of each rendered, final frame
  • An encoded preview video file with all the frames
  • The original input file(s) used for the session
  • Some logs, checksums and performance data


In that list the first item is typically the one using the most space. By using lossless compression it is possible to store the exact same data using less space. How much less differs a lot from session to session but it is typically between 5%-50%.

What you are seeing is the new CruncherService going through the backlog of all the old rendered sessions and performing two tasks: validating that the data is still correct and compressing rendered frames using lossless compression.
At the moment it has been granted very few CPU resources and it is running as a background service so it is running fairly slowly - but it will pick up in speed once the new server is in place. The process is fairly I/O heavy so we're keeping it on the server cluster instead of distributing it to the farm via BOINC.

17) Message boards : Comments and discussion : 1795 (Message 15236)
Posted 11 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, it has been rejected because it is a PNG image file and doesn't describe a 3D scene to be rendered in any of the currently supported renderers.
18) Message boards : Comments and discussion : 2730 (Message 15235)
Posted 11 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, it has been rejected because it is a JPEG file and doesn't describe a 3D scene to be rendered in any of the currently supported renderers.
19) Message boards : Comments and discussion : 1724 (Message 15234)
Posted 11 Sep 2017 by Profile Janus
This session got stuck in an unfortunate bug in the upload handler and was recently recovered.

However, this session has been rejected because we do not support rendering from .zip-files.
20) Message boards : Server backend and mirrors : Lots of old sessions on the server status page (Message 15233)
Posted 11 Sep 2017 by Profile Janus
You are very likely right, it may have been a contributing factor in why there has been so few sessions recently.

We're down to around 17 now. I have to admit that it kinda drowned a bit in some of the other exciting stuff going on with new server hardware planning and the BARF development milestone, which I'll have to write a more detailed post about soonish.

A few of them will probably be used to test a series of new client releases for Blender 2.79 in a few weeks when it officially launches. There's probably going to be more client tests than usual.

One of the reasons why they are being (painstakingly slowly) manually rejected is that the system that was originally responsible for bugging them out is still a planned feature and the orphaned sessions represent a great source of testing data - and a way to further fix and fine-tune that system. The intention is to (properly) enable the system at one point, so that it may sift through incoming sessions and very quickly point out common upload issues to their authors even before an admin reviews the session.


Next 20