wtorek, 3 października 2017

Reverse engineering the rendering of The Witcher 3, part 2 - eye adaptation

Hi everyone!

Welcome to the second part of my mini series where I demystify some rendering techniques from The Witcher 3. This time it's gonna be much, much simpler than before.

In the first part I showed you how tonemapping is done in TW3. While explaining theoretical basics, I briefly mentioned about eye adaptation. And guess what? Today I'll show how this eye adaptation is handled.

But wait, what is this eye adaptation all about and why do we need that? Wikipedia knows all about this, but let's imagine that you are in dark room (Life is Strange, anyone? :) ) or cave and you go outside, where is bright. The primary source of this brightness can be Sun, for instance.

In darkness our pupils are big to let more light through them to retinas. When it gets brightly, our pupils are becoming smaller and sometimes we blink, because it "hurts".
This change doesn't happen immediately. Eye has to adapt to changes of brightness. This is exactly why we perform eye adaptation in real time rendering.

Good example where lack of eye adaptation is noticeable is HDRToneMappingCS11 from DirectX SDK. Abrupt changes of average luminance are rather unpleasant and unnatural.

Let's get started!
For consistency, we will be analyzing the same frame as before, from Novigrad City.

And now some diving into RenderDoc frame capture. Eye adaptation is usually done just before tonemapping and The Witcher 3 is no exception.

And look at the pixel shader state:

We have 2 inputs - 2 textures, R32_FLOAT, 1x1 (one pixel).
texture0 contains average scene luminance from previous frame.
texture1 contains average scene luminance from current frame (computed just before in compute shader - I marked this in blue color).

Not surprisingly, 1 output, R32_FLOAT, 1x1.

Let's take a look at pixel shader.

    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb3[1], immediateIndexed  
    dcl_sampler s0, mode_default  
    dcl_sampler s1, mode_default  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_resource_texture2d (float,float,float,float) t1  
    dcl_output o0.xyzw  
    dcl_temps 1  
   0: sample_l(texture2d)(float,float,float,float) r0.x, l(0, 0, 0, 0), t1.xyzw, s1, l(0)  
   1: sample_l(texture2d)(float,float,float,float) r0.y, l(0, 0, 0, 0), t0.yxzw, s0, l(0)  
   2: ge r0.z, r0.y, r0.x  
   3: add r0.x, -r0.y, r0.x  
   4: movc r0.z, r0.z, cb3[0].x, cb3[0].y  
   5: mad o0.xyzw, r0.zzzz, r0.xxxx, r0.yyyy  
   6: ret  

Wow, so easy! Only 7 lines of assembly :)
What is going on here? Explanation line by line:

0) Get average luminance from current frame.
1) Get average luminance from previus frame.
2) Perform a test: Is the current luminance less than or equal to luminance from previous frame?
If yes - luminance is going down, if no - luminance is getting higher.
3) Calculate difference: difference = currentLum - previousLum.
4) This conditional move (movc) assignes speed factor from constant buffer. Depending on the test from line #2, two different values can be assigned. This is smart, because you can have different adaptation speeds for both falling and rising of luminance. But in every single frame I investiagated, both values are the same, ranging from about 0.11 to 0.3.
5) Final calculation of adapted Luminance:
   adaptedLuminance = speedFactor * difference + previousLuminance.
6) End of the shader

Simple enough to implement in HLSL:
 // The Witcher 3 eye adaptation shader  
 cbuffer cBuffer : register (b3)  
   float4 cb3_v0;  
   float4 Position                                             : SV_Position;  
 SamplerState samplerPointClamp : register (s0);  
 SamplerState samplerPointClamp2 : register (s1);  
 Texture2D TexPreviousAvgLuminance  : register (t0);  
 Texture2D TexCurrentAvgLuminance  : register (t1);  
 float4 TW3_EyeAdaptationPS(VS_OUTPUT_POSTFX Input) : SV_TARGET  
   // Get current and previous luminance.  
   float currentAvgLuminance = TexCurrentAvgLuminance.SampleLevel( samplerPointClamp2, float2(0.0, 0.0), 0 );  
   float previousAvgLuminance = TexPreviousAvgLuminance.SampleLevel( samplerPointClamp, float2(0.0, 0.0), 0 );  
   // Scale factor. Can be different for both falling down and rising up of luminance.  
   // It affects speed of adaptation.  
   // Small conditional test is performed here, so different speed can be set differently for both these cases.  
   float adaptationSpeedFactor = (currentAvgLuminance <= previousAvgLuminance) ? cb3_v0.x : cb3_v0.y;  
   // Calculate adapted luminance.  
   float adaptedLuminance = lerp( previousAvgLuminance, currentAvgLuminance, adaptationSpeedFactor );  
   return adaptedLuminance;  

It gives us the same assembly. I would suggest only changing output type to float instead of float4. No need to waste bandwidth.

So this is how eye adaptation is done in Witcher 3. Pretty easy, huh? :)
I hope you enjoyed this post! Stay tuned for more.

Edit - Decemeber 15, 2018
Hi, at the time of writing this post, I haven't recognized HLSL compiler patterns well enough to notice there is no need of writing "difference" and so on.
This is simply a linear interpolation, lerp/mix, you name it.
The way lerp(x, y, s) is performed on HLSL assembly is simply

 x + s(y-x). 

And the (y-x) difference has to be stored somewhere.

Have a good day,

PS. Huge thanks to Baldur Karlsson ( Twitter: @baldurk ) for RenderDoc. It simply rocks.

Brak komentarzy:

Prześlij komentarz