3411

Message boards : Comments and discussion : 3411
Message board moderation

To post messages, you must log in.

AuthorMessage
Professor Desty Nova
Avatar

Send message
Joined: 21 Mar 05
Posts: 97
Credit: 290,527
RAC: 0
Message 15282 - Posted: 12 Nov 2017, 15:00:31 UTC

This Session seems to be triggering lots of "Completed, marked as invalid" WU.


Professor Desty Nova
Researching Karma the Hard way
ID: 15282 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jon b.

Send message
Joined: 5 Oct 11
Posts: 26
Credit: 4,613,849
RAC: 1
Message 15283 - Posted: 12 Nov 2017, 19:22:38 UTC - in response to Message 15282.  

I'm getting an invalid rate of about 12.5% on these WUs.

My Windows hosts are also not receiving any work:

11/12/2017 12:19:49 PM | BURP | Sending scheduler request: To fetch work.
11/12/2017 12:19:49 PM | BURP | Requesting new tasks for CPU
11/12/2017 12:19:51 PM | BURP | Scheduler request completed: got 0 new tasks
11/12/2017 12:19:51 PM | BURP | No tasks sent
11/12/2017 12:19:51 PM | BURP | Tasks are committed to other platforms
ID: 15283 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4556
Credit: 2,097,282
RAC: 0
Message 15284 - Posted: 13 Nov 2017, 21:07:32 UTC
Last modified: 13 Nov 2017, 21:16:54 UTC

I believe this session is the perfect example on why the current pixel-based validator sometimes fails entirely on Cycles renders. If you click the picture in the preview and zoom in there's this very fine grained random noise all over. It is different every time a node returns a frame.

Good news is that it seems there is a way to turn that to an advantage. The pixels may not be the same but the noise characteristics from each render node almost is. By analyzing the noise in segments, comparing THAT, and then comparing the remaining pixel data in the rendered images with filtering for the noise profile applied it should be possible to get a lot closer to a good validator for Cycles sessions.

No idea when the implementation of this will be fully ready but we are currently storing all the returned variations of each frame in order to be able to test the implementation on that data when it is.

In parallel with the above idea there is another validation mode being tested already, it has to do with detecting large features in each image and matching those to each other. It is going to be interesting to see if those two ideas combined could make for a validator that works well for both Blender Internal (like the pixel validator) and Cycles.

Naturally the "marked as invalid" WUs will also be credited eventually
ID: 15284 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4556
Credit: 2,097,282
RAC: 0
Message 15298 - Posted: 20 Nov 2017, 18:14:12 UTC - in response to Message 15284.  
Last modified: 20 Nov 2017, 18:15:31 UTC

No idea when the implementation of this will be fully ready but we are currently storing all the returned variations of each frame in order to be able to test the implementation on that data when it is.


Currently working on this. Unfortunately it requires a rewrite of the validator service interface.
ID: 15298 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jon b.

Send message
Joined: 5 Oct 11
Posts: 26
Credit: 4,613,849
RAC: 1
Message 15303 - Posted: 29 Nov 2017, 19:00:29 UTC

Janus, I think this session could be completed faster if you could somehow increase the number of tasks available to Windows hosts. My Linux hosts are able to consistently get new work, but the Windows hosts are left idle most of the time. Since almost two-thirds of the hosts attached to this project are Windows hosts, this is a tremendous loss of computing power.
ID: 15303 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 21 Mar 17
Posts: 18
Credit: 508,435
RAC: 2
Message 15323 - Posted: 19 Dec 2017, 15:43:34 UTC

Still many invalids. It would have been completed a month ago if the tasks were invalid.
ID: 15323 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DangerNerd
Project donor

Send message
Joined: 31 Mar 06
Posts: 127
Credit: 512,980
RAC: 2
Message 15324 - Posted: 21 Dec 2017, 5:05:17 UTC

Hi there,

Hope you don't mind me posting this. I wanted to confirm that I too am being denied work based on it being committed to other platforms. As of this post, server status states 33 WU ready to send... Just not to my hosts apparently. :-)

I'm not upset or anything, but would like to help as much as possible.

Thanks for the continued work on my favorite BOINC project.


12/20/2017 10:59:43 PM | BURP | update requested by user
12/20/2017 10:59:48 PM | BURP | Sending scheduler request: Requested by user.
12/20/2017 10:59:48 PM | BURP | Requesting new tasks for CPU and NVIDIA GPU
12/20/2017 10:59:49 PM | BURP | Scheduler request completed: got 0 new tasks
12/20/2017 10:59:49 PM | BURP | No tasks sent
12/20/2017 10:59:49 PM | BURP | Tasks are committed to other platforms
Our Advice is to support all useful BOINC projects. Smart people needed to give advice to those who seek answers: Give or Get Free Advice Here
ID: 15324 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 21 Mar 17
Posts: 18
Credit: 508,435
RAC: 2
Message 15325 - Posted: 21 Dec 2017, 11:38:09 UTC - in response to Message 15324.  

Hi there,

Hope you don't mind me posting this. I wanted to confirm that I too am being denied work based on it being committed to other platforms. As of this post, server status states 33 WU ready to send... Just not to my hosts apparently. :-)

I'm not upset or anything, but would like to help as much as possible.

Thanks for the continued work on my favorite BOINC project.


12/20/2017 10:59:43 PM | BURP | update requested by user
12/20/2017 10:59:48 PM | BURP | Sending scheduler request: Requested by user.
12/20/2017 10:59:48 PM | BURP | Requesting new tasks for CPU and NVIDIA GPU
12/20/2017 10:59:49 PM | BURP | Scheduler request completed: got 0 new tasks
12/20/2017 10:59:49 PM | BURP | No tasks sent
12/20/2017 10:59:49 PM | BURP | Tasks are committed to other platforms


Linux is receiving plenty of work and you have 3 Windows computers attached. Any Linux computers available?
ID: 15325 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4556
Credit: 2,097,282
RAC: 0
Message 15326 - Posted: 21 Dec 2017, 17:25:44 UTC

Looking into why there is so few WUs being created... hopefully this should improve soon
ID: 15326 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 21 Mar 17
Posts: 18
Credit: 508,435
RAC: 2
Message 15327 - Posted: 24 Dec 2017, 16:33:03 UTC

6 invalid, 8 inconclusive on my 1950x. Batting 1000
ID: 15327 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JZD

Send message
Joined: 30 Dec 11
Posts: 100
Credit: 3,487,889
RAC: 0
Message 15337 - Posted: 4 Jan 2018, 10:02:41 UTC - in response to Message 15326.  

Beauty, so now it's just a windows session. The Linux user is very grateful to you. :-(

Server Status - Tasks by application

Application ----------------- Unsent ----------- In progress
Blender (CPU) ------------ 0 ------------------- 0
Blender (Windows) ----- 20 ------------------ 285
Blender (Linux) ----------- 0 ------------------- 0
ID: 15337 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
LumenDan
Project donor

Send message
Joined: 27 May 05
Posts: 58
Credit: 1,178,357
RAC: 1
Message 15338 - Posted: 6 Jan 2018, 3:24:25 UTC - in response to Message 15284.  
Last modified: 6 Jan 2018, 3:30:02 UTC

I believe this session is the perfect example on why the current pixel-based validator sometimes fails entirely on Cycles renders. If you click the picture in the preview and zoom in there's this very fine grained random noise all over. It is different every time a node returns a frame.

Good news is that it seems there is a way to turn that to an advantage. The pixels may not be the same but the noise characteristics from each render node almost is. By analyzing the noise in segments, comparing THAT, and then comparing the remaining pixel data in the rendered images with filtering for the noise profile applied it should be possible to get a lot closer to a good validator for Cycles sessions.

Could this noise-profiling technique be used to eliminate noise from a cycles render frame?
Given that each frame has at least one redundant copy from an alternative node, the pixels known to be noise could be replaced using pixels from a frame rendered on another node. If the result is still too noisy to validate pixel-by-pixel further replication would increase the accuracy of the resulting frame without requiring a massive cycles count and eventually lead to a combined consensus in validation.
I can see how this approach makes it harder to check for genuinely invalid frames where a node has produced a vastly different image to it's peers and I guess this is where the "large feature recognition" technique comes into play, quickly eliminating frames that shouldn't be combined in an attempt to increase the accuracy of a frame.
Would a multi pass pixel comparison be useful to identify images as being similar?

Pass1: Record the pixel coordinates of all pixels that do not match.

Pass2: Invalid pixel coordinates from the first pass are combined into regions, if the area of all error regions is very small (perhaps < 4px) then either return a valid comparison or mark the frame as inconclusive and replicate the work unit for the frame.

If using the noise profile of nodes to reduce errors is possible it would add value to the redundancy that we already have in distributed computing especially if each work unit replication has the potential to improve the resulting frame.
ID: 15338 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Comments and discussion : 3411