Reverse engineering the rendering of The Witcher 3, part 3 - chromatic aberration

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".


Hello!

Welcome to the third episode of my mini series where I demystify some rendering techniques from The Witcher 3.

Today we will take a closer look at chromatic aberration.

Chromatic aberration is an effect known mostly from cheaper lenses. It occurs because lenses have different refractive index for different wavelenghts of visible light. The result of this is visible distortion.

Not everyone likes it though. Luckily in The Witcher 3 this effect is very slight and therefore is not disturbing during gameplay  (at least for me). However, you can disable it if you want to.

Let's take a closer look at an example scene with and without chromatic aberration:

Chromatic aberration: on

Chromatic aberration: off

Okay, do you see any difference near the corners? Me neither. Let's try different scene:

Chromatic aberration: On (#2). Notice slight "red" distortion in marked region.

Ah! Much better! There is bigger contrast between dark and bright regions and in the corner we can see slight distortion.

As you can see, this effect is really slight. Anyway, I was curious how this was implemented.
So let's go now to the most interesting part: code!

Implementation
The first thing to do is to find proper draw call with pixel shader.
Actually, chromatic aberration is part of a bigger "final postprocess" pixel shader, which consists of chromatic aberration, vignette and gamma correction, all in one PS.

So let's take a closer look at pixel shader assembly:

 ps_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb3[18], immediateIndexed  
    dcl_sampler s1, mode_default  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_input_ps_siv v0.xy, position  
    dcl_input_ps linear v1.zw  
    dcl_output o0.xyzw  
    dcl_temps 4  
   0: mul r0.xy, v0.xyxx, cb3[17].zwzz  
   1: mad r0.zw, v0.xxxy, cb3[17].zzzw, -cb3[17].xxxy  
   2: div r0.zw, r0.zzzw, cb3[17].xxxy  
   3: dp2 r1.x, r0.zwzz, r0.zwzz  
   4: sqrt r1.x, r1.x  
   5: add r1.y, r1.x, -cb3[16].y  
   6: mul_sat r1.y, r1.y, cb3[16].z  
   7: sample_l(texture2d)(float,float,float,float) r2.xyz, r0.xyxx, t0.xyzw, s1, l(0)  
   8: lt r1.z, l(0), r1.y  
   9: if_nz r1.z  
  10:  mul r1.y, r1.y, r1.y  
  11:  mul r1.y, r1.y, cb3[16].x  
  12:  max r1.x, r1.x, l(0.000100)  
  13:  div r1.x, r1.y, r1.x  
  14:  mul r0.zw, r0.zzzw, r1.xxxx  
  15:  mul r0.zw, r0.zzzw, cb3[17].zzzw  
  16:  mad r0.xy, -r0.zwzz, l(2.000000, 2.000000, 0.000000, 0.000000), r0.xyxx  
  17:  sample_l(texture2d)(float,float,float,float) r2.x, r0.xyxx, t0.xyzw, s1, l(0)  
  18:  mad r0.xy, v0.xyxx, cb3[17].zwzz, -r0.zwzz  
  19:  sample_l(texture2d)(float,float,float,float) r2.y, r0.xyxx, t0.xyzw, s1, l(0)  
  20: endif  
 ...  

And cbuffer values:

Okay, let's try to understand what's going on here.

cb3_v17.xy is essentialy center of chromatic aberration, so the first lines are essentially calculating 2d vector from texel coords (cb3_v17.zw = inverse viewport size) to "chromatic aberration center" and its length, then some maths, test and branching.

When chromatic aberration is applied, we calculate offsets using some values from constant buffer and we distort R and G channels.

Generally, the closer to corners of screen, the more intense the effect is. Line 10 is quite an interesting one, because it makes pixels to "come closer", especially when we exaggerate the aberration.

And I'm pleased to share with you with my implementation of it. As always, please take names of variables with (large) grain of salt. And note this effect is done *prior* to gamma correction.

 void ChromaticAberration( float2 uv, inout float3 color )  
 {  
   // User-defined params  
   float2 chromaticAberrationCenter = float2(0.5, 0.5);  
   float chromaticAberrationCenterAvoidanceDistance = 0.2;  
   float fA = 1.25;  
   float fChromaticAbberationIntensity = 30;  
   float fChromaticAberrationDistortionSize = 0.75;  
   
   // Calculate vector  
   float2 chromaticAberrationOffset = uv - chromaticAberrationCenter;  
   chromaticAberrationOffset = chromaticAberrationOffset / chromaticAberrationCenter;  
     
   float chromaticAberrationOffsetLength = length(chromaticAberrationOffset);  
    
   // To avoid applying chromatic aberration in center, subtract small value from  
   // just calculated length.  
   float chromaticAberrationOffsetLengthFixed = chromaticAberrationOffsetLength - chromaticAberrationCenterAvoidanceDistance;  
   float chromaticAberrationTexel = saturate(chromaticAberrationOffsetLengthFixed * fA);  
   
   float fApplyChromaticAberration = (0.0 < chromaticAberrationTexel);  
   if (fApplyChromaticAberration)  
   {  
     chromaticAberrationTexel *= chromaticAberrationTexel;  
     chromaticAberrationTexel *= fChromaticAberrationDistortionSize;  
   
     chromaticAberrationOffsetLength = max(chromaticAberrationOffsetLength, 1e-4);  
       
     float fMultiplier = chromaticAberrationTexel / chromaticAberrationOffsetLength;  
   
     chromaticAberrationOffset *= fMultiplier;  
     chromaticAberrationOffset *= g_Viewport.zw;  
     chromaticAberrationOffset *= fChromaticAbberationIntensity;  
   
     float2 offsetUV = -chromaticAberrationOffset * 2 + uv;  
     color.r = TexColorBuffer.SampleLevel(samplerLinearClamp, offsetUV, 0).r;  
   
     offsetUV = uv - chromaticAberrationOffset;  
     color.g = TexColorBuffer.SampleLevel(samplerLinearClamp, offsetUV, 0).g;  
   }  
 }  

I've added "fChromaticAberrationIntensity" to increase size of offset, therefore, intensity of the effect, as name suggets (TW3 = 1.0).

Intensity = 40:

So this is it! I hope you have enjoyed this post.
Stay tuned for more, at least few more effects are waiting to be reverse engineered! :)

Have a good day,
M.

Reverse engineering the rendering of The Witcher 3, part 2 - eye adaptation

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".


Hi everyone!

Welcome to the second part of my mini series where I demystify some rendering techniques from The Witcher 3. This time it's gonna be much, much simpler than before.

In the first part I showed you how tonemapping is done in TW3. While explaining theoretical basics, I briefly mentioned about eye adaptation. And guess what? Today I'll show how this eye adaptation is handled.

But wait, what is this eye adaptation all about and why do we need that? Wikipedia knows all about this, but let's imagine that you are in dark room (Life is Strange, anyone? :) ) or cave and you go outside, where is bright. The primary source of this brightness can be Sun, for instance.

In darkness our pupils are big to let more light through them to retinas. When it gets brightly, our pupils are becoming smaller and sometimes we blink, because it "hurts".
This change doesn't happen immediately. Eye has to adapt to changes of brightness. This is exactly why we perform eye adaptation in real time rendering.

Good example where lack of eye adaptation is noticeable is HDRToneMappingCS11 from DirectX SDK. Abrupt changes of average luminance are rather unpleasant and unnatural.

Let's get started!
For consistency, we will be analyzing the same frame as before, from Novigrad City.

And now some diving into RenderDoc frame capture. Eye adaptation is usually done just before tonemapping and The Witcher 3 is no exception.

And look at the pixel shader state:

We have 2 inputs - 2 textures, R32_FLOAT, 1x1 (one pixel).
texture0 contains average scene luminance from previous frame.
texture1 contains average scene luminance from current frame (computed just before in compute shader - I marked this in blue color).

Not surprisingly, 1 output, R32_FLOAT, 1x1.

Let's take a look at pixel shader.

 ps_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb3[1], immediateIndexed  
    dcl_sampler s0, mode_default  
    dcl_sampler s1, mode_default  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_resource_texture2d (float,float,float,float) t1  
    dcl_output o0.xyzw  
    dcl_temps 1  
   0: sample_l(texture2d)(float,float,float,float) r0.x, l(0, 0, 0, 0), t1.xyzw, s1, l(0)  
   1: sample_l(texture2d)(float,float,float,float) r0.y, l(0, 0, 0, 0), t0.yxzw, s0, l(0)  
   2: ge r0.z, r0.y, r0.x  
   3: add r0.x, -r0.y, r0.x  
   4: movc r0.z, r0.z, cb3[0].x, cb3[0].y  
   5: mad o0.xyzw, r0.zzzz, r0.xxxx, r0.yyyy  
   6: ret  

Wow, so easy! Only 7 lines of assembly :)
What is going on here? Explanation line by line:

0) Get average luminance from current frame.
1) Get average luminance from previus frame.
2) Perform a test: Is the current luminance less than or equal to luminance from previous frame?
If yes - luminance is going down, if no - luminance is getting higher.
3) Calculate difference: difference = currentLum - previousLum.
4) This conditional move (movc) assignes speed factor from constant buffer. Depending on the test from line #2, two different values can be assigned. This is smart, because you can have different adaptation speeds for both falling and rising of luminance. But in every single frame I investiagated, both values are the same, ranging from about 0.11 to 0.3.
5) Final calculation of adapted Luminance:
   adaptedLuminance = speedFactor * difference + previousLuminance.
6) End of the shader

Simple enough to implement in HLSL:

 // The Witcher 3 eye adaptation shader  
   
 cbuffer cBuffer : register (b3)  
 {  
   float4 cb3_v0;  
 }
  
 struct VS_OUTPUT_POSTFX  
 {  
   float4 Position                                             : SV_Position;  
 };  
  
 SamplerState samplerPointClamp : register (s0);  
 SamplerState samplerPointClamp2 : register (s1);  
   
 Texture2D TexPreviousAvgLuminance  : register (t0);  
 Texture2D TexCurrentAvgLuminance  : register (t1);  
   
 float4 TW3_EyeAdaptationPS(VS_OUTPUT_POSTFX Input) : SV_TARGET  
 {  
   // Get current and previous luminance.  
   float currentAvgLuminance = TexCurrentAvgLuminance.SampleLevel( samplerPointClamp2, float2(0.0, 0.0), 0 );  
   float previousAvgLuminance = TexPreviousAvgLuminance.SampleLevel( samplerPointClamp, float2(0.0, 0.0), 0 );  
     
   // Scale factor. Can be different for both falling down and rising up of luminance.  
   // It affects speed of adaptation.  
   // Small conditional test is performed here, so different speed can be set differently for both these cases.  
   float adaptationSpeedFactor = (currentAvgLuminance <= previousAvgLuminance) ? cb3_v0.x : cb3_v0.y;  
   
   // Calculate adapted luminance.  
   float adaptedLuminance = lerp( previousAvgLuminance, currentAvgLuminance, adaptationSpeedFactor );  
   return adaptedLuminance;  
 }  

It gives us the same assembly. I would suggest only changing output type to float instead of float4. No need to waste bandwidth.

So this is how eye adaptation is done in Witcher 3. Pretty easy, huh? :)
I hope you enjoyed this post! Stay tuned for more.


Edit - Decemeber 15, 2018
Hi, at the time of writing this post, I haven't recognized HLSL compiler patterns well enough to notice there is no need of writing "difference" and so on.
This is simply a linear interpolation, lerp/mix, you name it.
The way lerp(x, y, s) is performed on HLSL assembly is simply

 x + s(y-x).

And the (y-x) difference has to be stored somewhere.


Have a good day,
M.

PS. Huge thanks to Baldur Karlsson ( Twitter: @baldurk ) for RenderDoc. It simply rocks.