Cycles Denoising on split frames


Advanced search

Message boards : Client : Cycles Denoising on split frames

Author Message
Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15293 - Posted: 20 Nov 2017, 2:58:57 UTC

Sheep it has disabled denoising for split frames because it would produce seams on the edges of the tiles.
How does BURP handle that at the moment ?
____________
https://caetano-veyssieres.com

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4492
Credit: 2,094,806
RAC: 0
Message 15294 - Posted: 20 Nov 2017, 7:32:14 UTC - in response to Message 15293.

Default policy is to not split frames if they use a feature (blur etc.) that could cause visual artifacts. In certain cases this means that people must resubmit their session in order to have it split.

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15295 - Posted: 20 Nov 2017, 13:15:10 UTC - in response to Message 15294.
Last modified: 20 Nov 2017, 13:33:12 UTC

I have this idea for avoiding seams when denoising on split frames, which is to add a margin on each tile and blend the margins between tiles gradually when combining them.
It sounds crazy but I don't think there is a better way nor will ever be.


If it sounds impossible to do, I think it can be made possible this way :

- The renderfarm's software would only add the margins (expand the borders on each tile, except for the actual camera borders which is neither possible nor needed) and once the rendering is finished, provide the tiles as they are for download.
- the session page would provide a link to a blend file containing the node setup to use for combining the tiles, so the user could do the compositing on his local machine.

What do you think ?
Do you have another idea ?
____________
https://caetano-veyssieres.com

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4492
Credit: 2,094,806
RAC: 0
Message 15297 - Posted: 20 Nov 2017, 17:13:03 UTC
Last modified: 20 Nov 2017, 17:15:32 UTC

Tried this already - it was very inefficient compared to simply rendering full-screen tiles (the overlap was rendered many times by many clients). It was especially difficult to establish just how much overlap was needed since some post-processing nodes require more overlap than others (and sometimes even the node settings are relevant too). Extreme blur, as used in the depth-of-field node for example, potentially requires a very large overlap.

In the long run split-frame rendering will have to do all post-processing on the server compute cluster from combined sample data from all clients. It is pretty much the only way to do this in a reasonable manner. We just aren't there yet, so at the moment we do full-frame rendering for that kind of scenes or require the artist to do the post-processing on their own machine after the split-frame render has been automatically combined by the server.

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15299 - Posted: 20 Nov 2017, 20:45:31 UTC - in response to Message 15297.
Last modified: 20 Nov 2017, 21:17:01 UTC

Ok but I was not talking about post-processing effects. Compositing locally is not a problem at all (as long as you download your frames in non destructive format, with high dynamic range and render passes).

What is problematic is the new denoiser in Blender 2.79.
This feature is not a post-processing feature, it's a rendering feature. You cannot use it after the render is done, it can only be used at render time.

For that reason, if we don't use some trick to prevent the seams, it can simply not be used on split frames, even locally after downloading the frames in EXR format.

However, contrarily to effects like blur which can vary drastically and unpredictably according to the values set in the nodes, denoising doesn't require much margin, provided that the artist used it the way it's supposed to be used, which is : remove the last bit of noise. Not smooth out a crazy noisy image.

Removing the last bit of noise with vs without the denoiser is a matter of going from 3K samples to 10K, which is a huge difference of render time (More on that here)

So if you use it correctly, you shouldn't need a margin bigger than a few pixels (like 4 or 5 per tile) to prevent seams.
____________
https://caetano-veyssieres.com

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15308 - Posted: 10 Dec 2017, 17:04:42 UTC - in response to Message 15293.
Last modified: 10 Dec 2017, 17:10:37 UTC

please delete this post
____________
https://caetano-veyssieres.com

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15310 - Posted: 10 Dec 2017, 19:53:44 UTC - in response to Message 15297.

Ok, let me wipe the dust from my good old "Camera Cropper & Splitter" and show you what I'm talking about.

With it I just made some render tests on split frames with denoising enabled. I looked for noticeable seams caused by the tiling messing with the denoiser, in order to see if they can be erased with a small blended margin... or even if there is anything to be erased at the first place.

After several tests with different scenes and various render settings, I found that the most noticeable seams appear on the simplest and smoothest scenes and with the lowest amount of samples (which is not what the denoiser was made for, as The Pixelary pointed out). But if you found better test cases than I did, please tell me.

Here, a smooth displacement on a plane with SSS, at a staggering 32 samples, split in 64 tiles (8x8) :



The seams are pretty visible but at such a low sampling amount, the regular denoising artifacts are even more visible. This is a non-destructive PNG image and it looks like a badly compressed jpeg. Here's the non-split version if you want to compare (Tip : use Ctrl+Tab and Ctrl+Shift+Tab to switch between images in tabs in Firefox)

And as we increase the amount of samples, the seams disappear quickly. Here it is at 128 samples :



There's only one small seam remaining at the middle-top but the other regular denoising artifacts are still pretty visible anyway.
(non-split version for comparison)

Once we get to 256 samples (which is still quite low, as this non-denoised version shows), the seams are completely gone :



non-split version

And that's not even taking 2 things into account :
One, "the denoiser is really designed to help you get rid of that last bit of noise. You know, the noise that forces you to render at 10000 samples rather than 3000" (The Pixelary).
And two, splitting a frame is usually meant for a decent resolution. 64 tiles was voluntarily ridiculously high for a 1024*768px image, to get the most obvious artifacts.

So what about more complex scenes and higher resolutions you ask ?

Here's the classroom benchmark, in 1080p, at the default - but still low - 300 samples.
Seams are very, very hard to find already :




If we get to decent amount of samples and reasonable resolution for final renders, seams are just out of question, even in the craziest situations.
Here are a bunch of motion blurred SSS spheres inside of a volume scattering fog with spotlights, at a reasonable 2048 samples and 2048x2048 pixels. Good luck finding any seam, and the overall sampling isn't even that clean yet.



So before I even get to fixing the seams, for what concerns me, we could simply support denoising on split frames with no worries. Not to mention that as soon as someone would get seams, that someone would very quickly understand that denoising is to blame. Even more so if - at the very same place of the upload page where it says "If you split frames, denoising will be disabled" - there was, instead, a warning telling us we might get seams at low sample values and low resolutions.


But let's fix the problem completely and erase those seams, right?

Let's get back to the ugliest image, the displaced SSS plane at 32 samples and 1k resolution and let's add a 4px margin on each tile and blend those margins gradually between the combined tiles:



No surprise, the seams are gone. You can compare with the first and second images of my post to see the difference.

If you still don't agree that denoising could at least be simply enabled on split frames, I guess I'll have to go back to work on my old-school Camera splitter add-on and make it easier to use for everyone for that use-case, but I'd prefer not to have to, as it's not nearly as convenient as a built-in feature in the render farm, which I believe would also save a lot of render time and free many computers from long full-frame render tasks that might get cancelled before they get finished.
____________
https://caetano-veyssieres.com

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4492
Credit: 2,094,806
RAC: 0
Message 15316 - Posted: 12 Dec 2017, 20:54:42 UTC - in response to Message 15310.

If you were to animate the 32sample,8x8 split plane with slightly overlapping tiles would the seams be more visible since they systematically deviate from the rest of the image by having samples from both tiles?

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15317 - Posted: 12 Dec 2017, 21:45:01 UTC - in response to Message 15316.
Last modified: 12 Dec 2017, 21:47:29 UTC

Here are 2 animation tests in lossless video encoding. I didn't do the displaced SSS plane in 32 samples with blended margins like you suggest yet but I will do it and post the result.

These don't have any blended margins. Just regular tiles with denoising enabled :



____________
https://caetano-veyssieres.com

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15319 - Posted: 14 Dec 2017, 12:56:32 UTC - in response to Message 15316.

Allright, here is the 32 samples animation test.
One is split in 8x8 tiles with a 4 pixels margin and the other is not split.

Now let's play a game called "which is which" ?
(the numbers in the filenames and their order of appearance were randomly assigned)

https://mega.nz/#!0HAQHLrR!rOFNy2TiTOU3UmWu_eP5dhH4vIkgKr9J_7NW0yN5SA8

https://mega.nz/#!ZOBWGJhA!5ozo7H_vhT95YJh5wvpbfnOcpE5E30NFqMtzi9yGIlU
____________
https://caetano-veyssieres.com

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4492
Credit: 2,094,806
RAC: 0
Message 15321 - Posted: 16 Dec 2017, 12:57:01 UTC - in response to Message 15319.
Last modified: 16 Dec 2017, 12:58:26 UTC

Not much of a difference really.

4 seems like a fairly arbitrary number. Would it look the same if you had used a nondeterministic sampling pattern with the 4pixel blending? We sometimes get back frames from Windows that are different from those on Linux. I could imagine that the blending zones could look odd in that case.

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15322 - Posted: 16 Dec 2017, 23:38:23 UTC - in response to Message 15321.
Last modified: 16 Dec 2017, 23:39:05 UTC

About the 4 pixel margin, to be more precise,it's 4 vertically and 4 * aspect ratio horizontally, since getting a 4 px margin on both dimensions would've been much trickier to do with my method.

About the sampling, not sure what you mean by nondeterministic, since from my undestanding, sobol and correlated multi-jitter are both deterministic and there's no 3rd option.

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15328 - Posted: 25 Dec 2017, 14:45:38 UTC - in response to Message 15321.

so what did you mean by nondeterministic ?
____________
https://caetano-veyssieres.com

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4492
Credit: 2,094,806
RAC: 0
Message 15335 - Posted: 2 Jan 2018, 12:52:18 UTC

Some sessions, and #3411 is a perfect example running right now, have a nondeterministic-random component somewhere. Individually rendered tile instances always come back with slightly different noise patterns. I was under the impression that the Sobol sampler was that component because the noise is all over the entire frame, but upon closer inspection the seed value is right there in the sampling settings and a quick simple test indeed shows that it is behaving quite fine at least on the test scene I made.

The issue I was hinting at was that, when you are in that situation, merging the extra margins would create an area in the frame that has the average of two different noise patterns (creating something less noisy), whereas the areas in the middle of a tile only have samples from one noise pattern. My suspicion is that this would be visible in the final result.

Profile ChameleonScales
Avatar
Send message
Joined: 2 Apr 16
Posts: 147
Credit: 1,120
RAC: 0
Message 15336 - Posted: 2 Jan 2018, 19:40:26 UTC - in response to Message 15335.
Last modified: 2 Jan 2018, 19:41:44 UTC

This would certainly be visible on non-denoised frames with low enough sample values, but I think the denoiser makes things different, because the artifacts of a denoised image are much more complex than the sampling noise. Although averaging may produce a cleaner image in some cases, it would only do it to a very limited point, contrarily to the sampling noise which can be cleaned indefinitely.

At least my render tests so far don't seem to show this "less noisy" areas in the margins, but we should do some render tests on averaging denoised images to see how the cleaning compares with the non-denoised averaged renders.

However, the margin's size would undoubtedly have to be more thought out and made dependent on the denoiser's parameters, especially the Denoising Radius.
____________
https://caetano-veyssieres.com


Post to thread

Message boards : Client : Cycles Denoising on split frames