Reverse engineering the rendering of The Witcher 3, part 9 - GBuffer

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".


Welcome,

This is the ninth part of my series about rendering in The Witcher 3. Click here for full index.

In this part I will show some details about geometry buffer (gbuffer) in The Witcher 3.

I assume here that you know the basics of deferred shading. 

Quick recap: the idea is to, well, defer rendering by not calculating all final lighting and shading immediately, but instead separate calculations into two stages.

In the first one (geometry pass) we fill GBuffer with data about surface (position, normals, specular color etc...) and in the second one (lighting pass) we combine everything and calculate lighting. 

Deferred shading is hugely popular approach because it allows to calculate lighting in one full-screen pass with techniques like tile-based deferred shading which greatly improves performance.

Simply speaking, GBuffer is collecton of textures with properties of geometry. It's very important to design its layout carefully. For real-life example, check for instance The Rendering Technologies of Crysis 3. 

After this brief introduction let's take a look at example frame from The Witcher 3: Blood & Wine:

One of many inns in Toussaint

The main GBuffer consists of three fullscreen render targets with DXGI_FORMAT_R8G8B8A8_UNORM format
and DXGI_FORMAT_D24_UNORM_S8_UINT depth+stencil buffer.

Here are screenshots of them:

Render Target 0 - RGB channels, surface color

Render Target 0 - A channel. I have no idea what it is, really.

Render Target 1 - RGB channels. We have normal vectors in [0-1] range here.

Render Target 1 - A channel. Looks like reflectance!

Render Target 2 - RGB channels. Looks like specular color!
A channel is black in this scene (but it is used later)

Depth buffer. Note that reversed depth is used here

Stencil buffer to mark certain type of pixels (like skin, vegetation etc)

This is not whole GBuffer. Lighting pass also uses reflection probes and other buffers but this is not the subject of this post.

Before I start the "main" part of this post, some general observations first:

General observations

1) The only buffer to clear is depth/stencil.

If you analyze aforementioned textures in any good frame analyzer you may be a little surprised, because there is no "Clear" call on them with exception of Depth/Stencil.

So in reality RenderTarget1 looks like this (notice "blurred" pixels on far plane):

This is simple and nice optimization. 

Take with you: ClearRenderTargetView calls are not free, so use them only when really necessary.

2) Reversed depth rocks

Many articles have been already written about precision of floating-point depth buffer. The Witcher 3 uses reversed-z which is natural choice for such game with open world and long draw distances.

For DirectX the switch shouldn't be difficult:

a) Clear depth buffer with "0" intead on "1".

In a traditional approach we used to clear depth buffer far value of "1". After reversing depth, the new "far" value is zero, so we need to change that.

b) Flip near and far clip values when calculating projection matrix

c) Change depth test from "Less" to "Greater".

For OpenGL there is a bit more work (see mentioned articles) but it is really worth the effort.

3) Do not store world position

It is that simple. Reconstruct world position from depth in lighting pass.

Pixel Shader

What I want to show in this post is pixel shader which feeds GBuffer with surface data. 

So we know by now we that store at least color, normals and specular.

Of course it's not that simple as you may think.

The problem with this pixel shader is that it comes in many variants. They differ in number of textures consumed and number of parameters used from constant buffer (probably constant buffer which describes material).

I decided to use this nice barrel for analyze:

Our heroic barrel!

And please give warm welcome to textures used:

So we have albedo, normal map and specular color. Pretty common scenario.

Before we start, few words about geometry inputs:
The geometry comes with position, texcoords, normal and tangent buffers.

Vertex Shader outputs at least texcoords, normalized tangent/normal/bitangent vectors multiplied earlier by world matrix. For more complicated materials (like with two diffuse or normal maps) vertex shader can output other data but I wanted to show here the simple cases.

Pixel Shader as assembly:

 ps_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb4[3], immediateIndexed  
    dcl_sampler s0, mode_default  
    dcl_sampler s13, mode_default  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_resource_texture2d (float,float,float,float) t1  
    dcl_resource_texture2d (float,float,float,float) t2  
    dcl_resource_texture2d (float,float,float,float) t13  
    dcl_input_ps linear v0.zw  
    dcl_input_ps linear v1.xyzw  
    dcl_input_ps linear v2.xyz  
    dcl_input_ps linear v3.xyz  
    dcl_input_ps_sgv v4.x, isfrontface  
    dcl_output o0.xyzw  
    dcl_output o1.xyzw  
    dcl_output o2.xyzw  
    dcl_temps 3  
   0: sample_indexable(texture2d)(float,float,float,float) r0.xyzw, v1.xyxx, t1.xyzw, s0  
   1: sample_indexable(texture2d)(float,float,float,float) r1.xyz, v1.xyxx, t0.xyzw, s0  
   2: add r1.w, r1.y, r1.x  
   3: add r1.w, r1.z, r1.w  
   4: mul r2.x, r1.w, l(0.333300)  
   5: add r2.y, l(-1.000000), cb4[1].x  
   6: mul r2.y, r2.y, l(0.500000)  
   7: mov_sat r2.z, r2.y  
   8: mad r1.w, r1.w, l(-0.666600), l(1.000000)  
   9: mad r1.w, r2.z, r1.w, r2.x  
  10: mul r2.xzw, r1.xxyz, cb4[0].xxyz  
  11: mul_sat r2.xzw, r2.xxzw, l(1.500000, 0.000000, 1.500000, 1.500000)  
  12: mul_sat r1.w, abs(r2.y), r1.w  
  13: add r2.xyz, -r1.xyzx, r2.xzwx  
  14: mad r1.xyz, r1.wwww, r2.xyzx, r1.xyzx  
  15: max r1.w, r1.z, r1.y  
  16: max r1.w, r1.w, r1.x  
  17: lt r1.w, l(0.220000), r1.w  
  18: movc r1.w, r1.w, l(-0.300000), l(-0.150000)  
  19: mad r1.w, v0.z, r1.w, l(1.000000)  
  20: mul o0.xyz, r1.wwww, r1.xyzx  
  21: add r0.xyz, r0.xyzx, l(-0.500000, -0.500000, -0.500000, 0.000000)  
  22: add r0.xyz, r0.xyzx, r0.xyzx  
  23: mov r1.x, v0.w  
  24: mov r1.yz, v1.zzwz  
  25: mul r1.xyz, r0.yyyy, r1.xyzx  
  26: mad r1.xyz, v3.xyzx, r0.xxxx, r1.xyzx  
  27: mad r0.xyz, v2.xyzx, r0.zzzz, r1.xyzx  
  28: uge r1.x, l(0), v4.x  
  29: if_nz r1.x  
  30:  dp3 r1.x, v2.xyzx, r0.xyzx  
  31:  mul r1.xyz, r1.xxxx, v2.xyzx  
  32:  mad r0.xyz, -r1.xyzx, l(2.000000, 2.000000, 2.000000, 0.000000), r0.xyzx  
  33: endif  
  34: sample_indexable(texture2d)(float,float,float,float) r1.xyz, v1.xyxx, t2.xyzw, s0  
  35: max r1.w, r1.z, r1.y  
  36: max r1.w, r1.w, r1.x  
  37: lt r1.w, l(0.200000), r1.w  
  38: movc r2.xyz, r1.wwww, r1.xyzx, l(0.120000, 0.120000, 0.120000, 0.000000)  
  39: add r2.xyz, -r1.xyzx, r2.xyzx  
  40: mad o2.xyz, v0.zzzz, r2.xyzx, r1.xyzx  
  41: lt r1.x, r0.w, l(0.330000)  
  42: mul r1.y, r0.w, l(0.950000)  
  43: movc r1.x, r1.x, r1.y, l(0.330000)  
  44: add r1.x, -r0.w, r1.x  
  45: mad o1.w, v0.z, r1.x, r0.w  
  46: dp3 r0.w, r0.xyzx, r0.xyzx  
  47: rsq r0.w, r0.w  
  48: mul r0.xyz, r0.wwww, r0.xyzx  
  49: max r0.w, abs(r0.y), abs(r0.x)  
  50: max r0.w, r0.w, abs(r0.z)  
  51: lt r1.xy, abs(r0.zyzz), r0.wwww  
  52: movc r1.yz, r1.yyyy, abs(r0.zzyz), abs(r0.zzxz)  
  53: movc r1.xy, r1.xxxx, r1.yzyy, abs(r0.yxyy)  
  54: lt r1.z, r1.y, r1.x  
  55: movc r1.xy, r1.zzzz, r1.xyxx, r1.yxyy  
  56: div r1.z, r1.y, r1.x  
  57: div r0.xyz, r0.xyzx, r0.wwww  
  58: sample_l(texture2d)(float,float,float,float) r0.w, r1.xzxx, t13.yzwx, s13, l(0)  
  59: mul r0.xyz, r0.wwww, r0.xyzx  
  60: mad o1.xyz, r0.xyzx, l(0.500000, 0.500000, 0.500000, 0.000000), l(0.500000, 0.500000, 0.500000, 0.000000)  
  61: mov o0.w, cb4[2].x  
  62: mov o2.w, l(0)  
  63: ret  

The shader has few stages. I will describe each main part of this shader separately.
But at first, as always - screenshot with values from constant buffer:

Albedo

We start with hard stuff. It's not that simple as "OutputColor.rgb = Texture.Sample(uv).rgb"
After we sample RGB of color texture (line 1) the next 14 lines are something which I called "desaturation filter". Let me show you HLSL code:

 float3 albedoColorFilter( in float3 color, in float desaturationFactor, in float3 desaturationValue )  
 {  
   float sumColorComponents = color.r + color.g + color.b;  
    
   float averageColorComponentValue = 0.3333 * sumColorComponents;  
   float oneMinusAverageColorComponentValue = 1.0 - averageColorComponentValue;  
     
   float factor = 0.5 * (desaturationFactor - 1.0);  
     
   float avgColorComponent = lerp(averageColorComponentValue, oneMinusAverageColorComponentValue, saturate(factor));  
   float3 desaturatedColor = saturate(color * desaturationValue * 1.5);  
    
   float mask = saturate( avgColorComponent * abs(factor) );  
   
   float3 finalColor = lerp( color, desaturatedColor, mask );  
   return finalColor;  
 }  

For majority of objects, this code does nothing but returns the original color from texture. This is achieved by proper "material cbuffer" values. cb4_v1.x is set to 1.0 which returns in mask equal to 0.0 and gives input color from lerp instruction.

However, there are some exceptions. The highest value of desaturationFactor I found was 4.0 (never below 1.0) and desaturatedColor depends on material. It can be something like (0.2, 0.3, 0.4); there are no strict rules. Of course I couldn't resist to implement this in my own DX11 framework and here are the results, all with desaturatedColor equal to float3( 0.25, 0.3, 0.45 )

desaturationFactor = 1.0 (no effect)

desaturationFactor = 2.0

desaturationFactor = 3.0

desaturationFactor = 4.0

I'm sure it's just applying material parameters but it's not the end of the albedo part.
Lines 15-20 perform final touches:

  15: max r1.w, r1.z, r1.y   
  16: max r1.w, r1.w, r1.x   
  17: lt r1.w, l(0.220000), r1.w   
  18: movc r1.w, r1.w, l(-0.300000), l(-0.150000)   
  19: mad r1.w, v0.z, r1.w, l(1.000000)   
  20: mul o0.xyz, r1.wwww, r1.xyzx   

v0.z is output from Vertex Shader and it's equal to zero. Remember it, because v0.z will be used later a couple of times.

It seems to be some factor and all this code looks like darkening albedo a little bit, but since v0.z is equal to 0, the color is untouched. HLSL:

   /* ALBEDO */  
   // optional desaturation (?) filter  
   float3 albedoColor = albedoColorFilter( colorTex, cb4_v1.x, cb4_v0.rgb );  
   float albedoMaxComponent = getMaxComponent( albedoColor );  
     
   // I really have no idea what this is  
   // In most of cases Vertex Shader outputs "paramZ" as 0  
   float paramZ = Input.out0.z;  // note, mostly 0  
   
   // Note that 0.70 are 0.85 are not present in the output assembly  
   // Because I wanted to use lerp here I had to adjust them manually.  
   float param = (albedoMaxComponent > 0.22) ? 0.70 : 0.85;  
   float mulParam = lerp(1, param, paramZ);  
   
   // Output  
   pout.RT0.rgb = albedoColor * mulParam;  
   pout.RT0.a = cb4_v2.x;  

Regarding RT0.a, as you can see, it comes from materal's constant buffer but since the shader has no debug information, it's hard to say exactly what this is. Maybe translucency?

We are done with the first render target!

Normals

We start by unpacking normal map, then we perform normal mapping as usual:

   /* NORMALS */   
   float3 sampledNormal = ((normalTex.xyz - 0.5) * 2);  
   
   // Data to construct TBN matrix  
   float3 Tangent = Input.TangentW.xyz;  
   float3 Normal = Input.NormalW.xyz;  
   float3 Bitangent;  
   Bitangent.x = Input.out0.w;  
   Bitangent.yz = Input.out1.zw;  
   
   // remove this saturate in real scenario, this is a hack to make sure normal-tbn multiplication  
   // will have 'mad' instructions in assembly instead a bunch of 'mov's
   Bitangent = saturate(Bitangent);  
     
   float3x3 TBN = float3x3(Tangent, Bitangent, Normal);  
   float3 normal = mul( sampledNormal, TBN );  

Nothing really surprising so far.

Take a look at lines 28-33:

  28: uge r1.x, l(0), v4.x   
  29: if_nz r1.x   
  30: dp3 r1.x, v2.xyzx, r0.xyzx   
  31: mul r1.xyz, r1.xxxx, v2.xyzx   
  32: mad r0.xyz, -r1.xyzx, l(2.000000, 2.000000, 2.000000, 0.000000), r0.xyzx   
  33: endif   

We can roughly write it this way:

   [branch] if (bIsFrontFace <= 0)  
   {  
      float cosTheta = dot(Input.NormalW, normal);  
      float3 invNormal = cosTheta * Input.NormalW;  
      normal = normal - 2*invNormal;  
   }  

I'm not sure if this is a proper way of writing this. If you know what type of mathematical operation this is - let me know.

We see that the pixel shader uses SV_IsFrontFace.
What's that? Documentation (I wanted to write 'msdn' but..) comes to the rescue:

"Specifies whether a triangle is front facing. For lines and points, IsFrontFace has the value true. The exception is lines drawn out of triangles (wireframe mode), which sets IsFrontFace the same way as rasterizing the triangle in solid mode. Can be written to by the geometry shader, and read by the pixel shader."

I also wanted to check it for myself. Indeed, the effect is visible in wireframe mode only. I believe the purpose of this piece of code is to properly calculate normals (therefore, lighting) in wireframe mode.
Here is a comparison: Both wireframe final scene color with this trick off/on as well as gbuffer normal [0-1] texture with this trick off/on:

Scene color without the trick

Scene color with the trick

Normals [0-1] without the trick

Normals [0-1] with the trick

Have you noticed that the format of every rendertarget of GBuffer is R8G8B8A8_UNORM? That means we have 256 possible values per one component. Is it enough for storing normals?

Storing high quality normals with reasonable amount of bytes in GBuffer is known problem but fortunately there is a lot of material to learn from.

Probably some of you already know what technique is used here. I'd like to say that in whole geometry pass there is one additional texture attached to slot #13...:

Ha! The Witcher 3 uses technique known as "Best Fit Normals". I will not go here in details (refer to the presentation). It was invented around 2009-2010 by Crytek and since CryEngine is open source, BFN is open source too.

BFN causes "grainy" look of normals texture.
Afer scaling normal with the best fit, we encode it from [-1;1] to [0, 1] range.

Specular 

We start from line 34, by sampling specular texture:

  34: sample_indexable(texture2d)(float,float,float,float) r1.xyz, v1.xyxx, t2.xyzw, s0   
  35: max r1.w, r1.z, r1.y   
  36: max r1.w, r1.w, r1.x   
  37: lt r1.w, l(0.200000), r1.w   
  38: movc r2.xyz, r1.wwww, r1.xyzx, l(0.120000, 0.120000, 0.120000, 0.000000)   
  39: add r2.xyz, -r1.xyzx, r2.xyzx   
  40: mad o2.xyz, v0.zzzz, r2.xyzx, r1.xyzx   

As you can see, there is similar "darkening" filter as with Albedo:
Calc component with max value, then calulate "darker" color and interpolate with original specular color using a parameter from vertex shader... which is set to 0, so we output color from texture.

HLSL:

   /* SPECULAR */  
   float3 specularTex = texture2.Sample( samplerAnisoWrap, Texcoords ).rgb;  
   
   // Similar algorithm as in Albedo. Calculate max component, compare this with  
   // some threshold and calculate "minimum" value if needed.  
   // Because in the scene I analyzed paramZ was set to zero, value from texture will be  
   // the final result.  
   float specularMaxComponent = getMaxComponent( specularTex );  
   float3 specB = (specularMaxComponent > 0.2) ? specularTex : float3(0.12, 0.12, 0.12);  
   float3 finalSpec = lerp(specularTex, specB, paramZ);  
   pout.RT2.xyz = finalSpec;  

Reflectivity

I have no idea if this name is proper for this parameter since I don't know how it affects lighting pass. The thing is that alpha channel of input normal map has additional data:

Alpha channel of "normal map" texture. (c) CD Projekt Red

Assembly:

  41: lt r1.x, r0.w, l(0.330000)   
  42: mul r1.y, r0.w, l(0.950000)   
  43: movc r1.x, r1.x, r1.y, l(0.330000)   
  44: add r1.x, -r0.w, r1.x   
  45: mad o1.w, v0.z, r1.x, r0.w   

Say hello to our old friend, 'v0.z'! This is similar to both albedo and specular:

   /* REFLECTIVITY */  
   float reflectivity = normalTex.a;  
   float reflectivity2 = (reflectivity < 0.33) ? (reflectivity * 0.95) : 0.33;  
     
   float finalReflectivity = lerp(reflectivity, reflectivity2, paramZ);  
   pout.RT1.a = finalReflectivity;  

Nice! This is the end of analyzing the first variant of pixel shader.

In terms of result, here is a comparison of my shader (left) with the original one (right):

These differences do not affect calculations so my job is done here ;)

Pixel Shader - "Albedo + Normals" variant

I decided to show you one more variant - now with albedo & normal maps only - without specular texture. The assembly is a bit longer:

 ps_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb4[8], immediateIndexed  
    dcl_sampler s0, mode_default  
    dcl_sampler s13, mode_default  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_resource_texture2d (float,float,float,float) t1  
    dcl_resource_texture2d (float,float,float,float) t13  
    dcl_input_ps linear v0.zw  
    dcl_input_ps linear v1.xyzw  
    dcl_input_ps linear v2.xyz  
    dcl_input_ps linear v3.xyz  
    dcl_input_ps_sgv v4.x, isfrontface  
    dcl_output o0.xyzw  
    dcl_output o1.xyzw  
    dcl_output o2.xyzw  
    dcl_temps 4  
   0: mul r0.x, v0.z, cb4[0].x  
   1: sample_indexable(texture2d)(float,float,float,float) r1.xyzw, v1.xyxx, t1.xyzw, s0  
   2: sample_indexable(texture2d)(float,float,float,float) r0.yzw, v1.xyxx, t0.wxyz, s0  
   3: add r2.x, r0.z, r0.y  
   4: add r2.x, r0.w, r2.x  
   5: add r2.z, l(-1.000000), cb4[2].x  
   6: mul r2.yz, r2.xxzx, l(0.000000, 0.333300, 0.500000, 0.000000)  
   7: mov_sat r2.w, r2.z  
   8: mad r2.x, r2.x, l(-0.666600), l(1.000000)  
   9: mad r2.x, r2.w, r2.x, r2.y  
  10: mul r3.xyz, r0.yzwy, cb4[1].xyzx  
  11: mul_sat r3.xyz, r3.xyzx, l(1.500000, 1.500000, 1.500000, 0.000000)  
  12: mul_sat r2.x, abs(r2.z), r2.x  
  13: add r2.yzw, -r0.yyzw, r3.xxyz  
  14: mad r0.yzw, r2.xxxx, r2.yyzw, r0.yyzw  
  15: max r2.x, r0.w, r0.z  
  16: max r2.x, r0.y, r2.x  
  17: lt r2.x, l(0.220000), r2.x  
  18: movc r2.x, r2.x, l(-0.300000), l(-0.150000)  
  19: mad r0.x, r0.x, r2.x, l(1.000000)  
  20: mul o0.xyz, r0.xxxx, r0.yzwy  
  21: add r0.xyz, r1.xyzx, l(-0.500000, -0.500000, -0.500000, 0.000000)  
  22: add r0.xyz, r0.xyzx, r0.xyzx  
  23: mov r1.x, v0.w  
  24: mov r1.yz, v1.zzwz  
  25: mul r1.xyz, r0.yyyy, r1.xyzx  
  26: mad r0.xyw, v3.xyxz, r0.xxxx, r1.xyxz  
  27: mad r0.xyz, v2.xyzx, r0.zzzz, r0.xywx  
  28: uge r0.w, l(0), v4.x  
  29: if_nz r0.w  
  30:  dp3 r0.w, v2.xyzx, r0.xyzx  
  31:  mul r1.xyz, r0.wwww, v2.xyzx  
  32:  mad r0.xyz, -r1.xyzx, l(2.000000, 2.000000, 2.000000, 0.000000), r0.xyzx  
  33: endif  
  34: add r0.w, -r1.w, l(1.000000)  
  35: log r1.xyz, cb4[3].xyzx  
  36: mul r1.xyz, r1.xyzx, l(2.200000, 2.200000, 2.200000, 0.000000)  
  37: exp r1.xyz, r1.xyzx  
  38: mad r0.w, r0.w, cb4[4].x, cb4[5].x  
  39: mul_sat r1.xyz, r0.wwww, r1.xyzx  
  40: log r1.xyz, r1.xyzx  
  41: mul r1.xyz, r1.xyzx, l(0.454545, 0.454545, 0.454545, 0.000000)  
  42: exp r1.xyz, r1.xyzx  
  43: max r0.w, r1.z, r1.y  
  44: max r0.w, r0.w, r1.x  
  45: lt r0.w, l(0.200000), r0.w  
  46: movc r2.xyz, r0.wwww, r1.xyzx, l(0.120000, 0.120000, 0.120000, 0.000000)  
  47: add r2.xyz, -r1.xyzx, r2.xyzx  
  48: mad o2.xyz, v0.zzzz, r2.xyzx, r1.xyzx  
  49: lt r0.w, r1.w, l(0.330000)  
  50: mul r1.x, r1.w, l(0.950000)  
  51: movc r0.w, r0.w, r1.x, l(0.330000)  
  52: add r0.w, -r1.w, r0.w  
  53: mad o1.w, v0.z, r0.w, r1.w  
  54: lt r0.w, l(0), cb4[7].x  
  55: and o2.w, r0.w, l(0.064706)  
  56: dp3 r0.w, r0.xyzx, r0.xyzx  
  57: rsq r0.w, r0.w  
  58: mul r0.xyz, r0.wwww, r0.xyzx  
  59: max r0.w, abs(r0.y), abs(r0.x)  
  60: max r0.w, r0.w, abs(r0.z)  
  61: lt r1.xy, abs(r0.zyzz), r0.wwww  
  62: movc r1.yz, r1.yyyy, abs(r0.zzyz), abs(r0.zzxz)  
  63: movc r1.xy, r1.xxxx, r1.yzyy, abs(r0.yxyy)  
  64: lt r1.z, r1.y, r1.x  
  65: movc r1.xy, r1.zzzz, r1.xyxx, r1.yxyy  
  66: div r1.z, r1.y, r1.x  
  67: div r0.xyz, r0.xyzx, r0.wwww  
  68: sample_l(texture2d)(float,float,float,float) r0.w, r1.xzxx, t13.yzwx, s13, l(0)  
  69: mul r0.xyz, r0.wwww, r0.xyzx  
  70: mad o1.xyz, r0.xyzx, l(0.500000, 0.500000, 0.500000, 0.000000), l(0.500000, 0.500000, 0.500000, 0.000000)  
  71: mov o0.w, cb4[6].x  
  72: ret

The differences between this variant and previous one are:

a) lines 1, 19: interpolation parameter v0.z is multiplied by cb4[0].x from constant buffer, but this product is used only to interpolate albedo at line 19. For other output data, 'usual' v0.z is used.


b) lines 54-55: o2.w is now set under condition that ( cb4[7].x > 0.0 )

We already know this pattern "someComparison - and" from calculating luminance histogram from TW3, we can write this as:

 pout.RT2.w = (cb4_v7.x > 0.0) ? (16.5/255.0) : 0.0;  

c) lines 34-42: completely different calculation of specular.

There is no specular texture. Let's see assembly responsible for that:

  34: add r0.w, -r1.w, l(1.000000)   
  35: log r1.xyz, cb4[3].xyzx   
  36: mul r1.xyz, r1.xyzx, l(2.200000, 2.200000, 2.200000, 0.000000)   
  37: exp r1.xyz, r1.xyzx   
  38: mad r0.w, r0.w, cb4[4].x, cb4[5].x   
  39: mul_sat r1.xyz, r0.wwww, r1.xyzx   
  40: log r1.xyz, r1.xyzx   
  41: mul r1.xyz, r1.xyzx, l(0.454545, 0.454545, 0.454545, 0.000000)   
  42: exp r1.xyz, r1.xyzx   

Note we used here (1-reflectivity). Luckily, this is quite simple in HLSL:

   float oneMinusReflectivity = 1.0 - normalTex.a;  
   float3 specularTex = pow(cb4_v3.rgb, 2.2);  
   oneMinusReflectivity = oneMinusReflectivity * cb4_v4.x + cb4_v5.x;  
   specularTex = saturate(specularTex * oneMinusReflectivity);  
   specularTex = pow(specularTex, 1.0/2.2);  
   
   // proceed as in the first variant...  
   float specularMaxComponent = getMaxComponent( specularTex ); 
   ... 

On a side note, in this variant we have slightly larger constant buffer with material data. These extra values are used to emulate specular color here.

The rest of the shader is the same as in prevous variant.

72 lines of assembly is a little too much for WinMerge to display at once so just believe me it's almost the same assembly as in original. Or you can grab my HLSLexplorer and see it for yourself! ;)

Summary

...and if you've come this far, maybe you're willing to come a little further.

Nothing what seems simple is not in real life and feeding the gbuffer in The Witcher 3 is no exception. I've just shown you the simplest variants of pixel shaders responsible for it and some general observations which apply to deferred shading in general.

For the most patient ones (or vice versa) the two variants of pixel shaders @ pastebin:

Variant 1 - with specular texture

Variant 2 - without specular texture

Feel free to comment.

I hope you enjoyed it.
Thanks for reading!

Reverse engineering the rendering of The Witcher 3, part 8 - The Moon and lunar phases

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".


Welcome,

In the 8th part of this series I will investigate the Moon shader from The Witcher 3 (more specifically, from "Blood and Wine" expansion pack).

The Moon is an important element of night sky and can be quite challenging to make it believable, but in TW3 for me it's just a pleasure to walk around during the night.
Just take a look at this scene!

Before I will get to the pixel shader, few words about rendering nuances. In terms of geometry it's just a sphere (see below) which comes with texture coordinates, normal and tangent vectors. The vertex shader calculates world space position as well as normalized normal, tangent, and bitangent (using cross product) vectors multiplied by world matrix.
To make sure that the Moon lies completely on far plane, MinDepth and MaxDepth fields of D3D11_VIEWPORT structure are set to 0.0 (the same trick is used for skydome). The Moon is rendered just after sky.

Sphere used to draw the Moon

Alright, I think we are ready to go. Let's see the pixel shader:

 ps_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb0[1], immediateIndexed  
    dcl_constantbuffer cb2[3], immediateIndexed  
    dcl_constantbuffer cb12[267], immediateIndexed  
    dcl_sampler s0, mode_default  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_input_ps linear v1.w  
    dcl_input_ps linear v2.xyzw  
    dcl_input_ps linear v3.xy  
    dcl_input_ps linear v4.xy  
    dcl_output o0.xyzw  
    dcl_temps 3  
   0: mov r0.x, -cb0[0].w  
   1: mov r0.y, l(0)  
   2: add r0.xy, r0.xyxx, v2.xyxx  
   3: sample_indexable(texture2d)(float,float,float,float) r0.xyzw, r0.xyxx, t0.xyzw, s0  
   4: add r0.xyz, r0.xyzx, l(-0.500000, -0.500000, -0.500000, 0.000000)  
   5: log r0.w, r0.w  
   6: mul r0.w, r0.w, l(2.200000)  
   7: exp r0.w, r0.w  
   8: add r0.xyz, r0.xyzx, r0.xyzx  
   9: dp3 r1.x, r0.xyzx, r0.xyzx  
  10: rsq r1.x, r1.x  
  11: mul r0.xyz, r0.xyzx, r1.xxxx  
  12: mul r1.xy, r0.yyyy, v3.xyxx  
  13: mad r0.xy, v4.xyxx, r0.xxxx, r1.xyxx  
  14: mad r0.xy, v2.zwzz, r0.zzzz, r0.xyxx  
  15: mad r0.z, cb0[0].y, l(0.033864), cb0[0].w  
  16: mul r0.z, r0.z, l(6.283185)  
  17: sincos r1.x, r2.x, r0.z  
  18: mov r2.y, r1.x  
  19: dp2_sat r0.x, r0.xyxx, r2.xyxx  
  20: mul r0.xyz, r0.xxxx, cb12[266].xyzx  
  21: mul r0.xyz, r0.xyzx, r0.wwww  
  22: mul r0.xyz, r0.xyzx, cb2[2].xyzx  
  23: add_sat r0.w, -v1.w, l(1.000000)  
  24: mul r0.w, r0.w, cb2[2].w  
  25: mul o0.xyz, r0.wwww, r0.xyzx  
  26: mov o0.w, l(0)  
  27: ret  

The main reason I selected shader from "Blood and Wine" expansions pack is simple - it's shorter ;)

At first we calculate offset for texture sampling.
cb0[0].w is used as offset along X axis. Using this simple trick we can simulate rotation of the Moon along its axis.

Example values from constant buffer

There is one texture  (1024x512) attached as input. We have normal map encoded in RGB channels and in alpha channel - color of the Moon's surface. Smart!

Alpha channel of the texture - color of the Moon's surface. (c) CD Projekt Red

RGB channels of the texture - normal map. (c) CD Projekt Red

Once we have proper texture coordinates, we sample RGBA channels. We have to unpack normal map and perform gamma correction of surface color. So far our HLSL shader can be written for example like this:

 float4 MoonPS(in InputStruct IN) : SV_Target0  
 {  
   // Texcoords offset  
   float2 uvOffsets = float2(-cb0_v0.w, 0.0);  
     
   // Final texcoords  
   float2 uv = IN.param2.xy + uvOffsets;  
   
   // Sample texture  
   float4 sampledTexture = texture0.Sample( sampler0, uv);  
   
   // Moon surface color - perform gamma correction  
   float moonColorTex = pow(sampledTexture.a, 2.2 );  
   
   // Unpack normal from [0,1] to [-1,1] range.  
   // Note: sampledTexture.xyz * 2.0 - 1.0 works the same way  
   float3 sampledNormal = normalize((sampledTexture.xyz - 0.5) * 2);  

The next is step is to perform normal mapping, but only on XY components. (In The Witcher 3, Z-axis is up and whole Z channel of the texture is 1.0) . We can do it like this:

   // Tangent space vectors  
   float3 Tangent = IN.param4.xyz;  
   float3 Normal = float3(IN.param2.zw, IN.param3.w);  
   float3 Bitangent = IN.param3.xyz;  
        
   // TBN matrix   
   float3x3 TBN = float3x3(Tangent, Bitangent, Normal);  
        
   // Calculate XY normal vector  
   // Squeeze TBN matrix to float3x2: 3 rows, 2 columns  
   float2 vNormal = mul(sampledNormal, (float3x2)TBN).xy;  

Now it's time for my favourite part of this shader. Take a look at lines 15-16 again:

  15: mad r0.z, cb0[0].y, l(0.033864), cb0[0].w  
  16: mul r0.z, r0.z, l(6.283185)

Well, what's this mysterious 0.033864? It seems to make no sense at first sight, but if we calculate its reciprocal, we'll get ~29.53, which is length of synodic month in days! Now this is what I call attention to detail!
We can safely assume that cb0[0].y is number of days which passed during gameplay. Additional bias which was used as X-axis offset of texture is used here.

Once we have this ratio, we multiply it by 2*Pi.
Then, using sincos, we calculate another 2d vector.

By calculating dot product between normal vector and "lunar" one lunar phase is simulated.

   // Lunar phase.  
   // We calculate days/29.53 + bias.  
   float phase = cb0_v0.y * (1.0 / SYNODIC_MONTH_LENGTH) + cb0_v0.w;  
   
   // Multiply by 2*PI. This way 29.53 will be a full period  
   // for sin/cos functions.  
   phase *= TWOPI;  
        
   // Calculate sine and cosine of lunar phase.  
   float outSin = 0.0;  
   float outCos = 0.0;  
   sincos(phase, outSin, outCos);  
        
   // Calculate lunar phase  
   float lunarPhase = saturate( dot(vNormal, float2(outCos, outSin)) );  

See some screenshots with various lunar phases:

The last step is to perform a series of multiplications to calculate final color.

   // Perform a series of multiplications to calculate final color.  
   
   // cb12_v266.xyz is used to boost Moon's glow and color.  
   // for example (1.54, 2.82, 4.13)  
   float3 moonSurfaceGlowColor = cb12_v266.xyz;  
   
   float3 moonColor = lunarPhase * moonSurfaceGlowColor;  
   moonColor = moonColorTex * moonColor;  
     
   // cb_v2.xyz is probably a filter, like (1.0, 1.0, 1.0)  
   moonColor *= cb2_v2.xyz;  
        
   // I'm not really sure what this thing is, maybe some horizon opacity value.  
   // Anyway, it doesn't seem to have that much influence to final color  
   // as parameters above.  
   float paramHorizon = saturate(1.0 - IN.param1.w);  
   paramHorizon *= cb2_v2.w;  
        
   moonColor *= paramHorizon;  
   
   // Output final color with zero alpha  
   return float4(moonColor, 0.0);  

You may wonder why this shader outputs 0.0 alpha. Well, the Moon is rendered with blending enabled:

Such approach allows us to have background (sky) color if this shader returns black one.

If you are interested in full shader, it's here. It has some big constant buffers and should be ready to inject instead original one in RenderDoc (just rename "MoonPS" to "EditedShaderPS").

Last but not least, I wanted to share results with you:
On the left - my shader, on the right - original shader from the game.
The difference is really minor which has no impact on results.

As you can see, this shader was quite easy to reconstruct.
I hope you enjoyed it.

Thanks for reading!

Reverse engineering the rendering of The Witcher 3, part 7b - average luminance (calculation)

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".


Welcome,

This is the second part of demystifying calculating of average luminance in "The Witcher 3: Wild Hunt". Being familiar with the first part is highly recommended.

Before we start the battle with another compute shader, let's do a quick recap of what happened previously: We were working on 1/4x1/4 downscaled HDR color buffer. What we have after the first pass is histogram of luminance (structured buffer of 256 unsigned integers). We calculated logarithm of each pixel's luma, distributed it across 256 cells and increased corresponding value of structured buffer by 1 per pixel. This way, total sum of all values in these 256 cells is equal to the number of pixels.

Example output of the first pass. There are 256 elements here.

For instance, our fullscreen buffer is 1920x1080. After downscaling, the first pass used 480x270 buffer. Sum of all 256 values in the buffer would be 480 * 270 = 129 600.

After this brief introduction, we're all ready to go to the next stage: calculation.
This time only one thread group is dispatched ( Dispatch(1, 1, 1) ).

Let's see the assembly of the compute shader:

 cs_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb0[1], immediateIndexed  
    dcl_uav_structured u0, 4  
    dcl_uav_typed_texture2d (float,float,float,float) u1  
    dcl_input vThreadIDInGroup.x  
    dcl_temps 4  
    dcl_tgsm_structured g0, 4, 256  
    dcl_thread_group 64, 1, 1  
   0: ld_structured_indexable(structured_buffer, stride=4)(mixed,mixed,mixed,mixed) r0.x, vThreadIDInGroup.x, l(0), u0.xxxx  
   1: store_structured g0.x, vThreadIDInGroup.x, l(0), r0.x  
   2: iadd r0.xyz, vThreadIDInGroup.xxxx, l(64, 128, 192, 0)  
   3: ld_structured_indexable(structured_buffer, stride=4)(mixed,mixed,mixed,mixed) r0.w, r0.x, l(0), u0.xxxx  
   4: store_structured g0.x, r0.x, l(0), r0.w  
   5: ld_structured_indexable(structured_buffer, stride=4)(mixed,mixed,mixed,mixed) r0.x, r0.y, l(0), u0.xxxx  
   6: store_structured g0.x, r0.y, l(0), r0.x  
   7: ld_structured_indexable(structured_buffer, stride=4)(mixed,mixed,mixed,mixed) r0.x, r0.z, l(0), u0.xxxx  
   8: store_structured g0.x, r0.z, l(0), r0.x  
   9: sync_g_t  
  10: if_z vThreadIDInGroup.x  
  11:  mul r0.x, cb0[0].y, cb0[0].x  
  12:  ftou r0.x, r0.x  
  13:  utof r0.y, r0.x  
  14:  mul r0.yz, r0.yyyy, cb0[0].zzwz  
  15:  ftoi r0.yz, r0.yyzy  
  16:  iadd r0.x, r0.x, l(-1)  
  17:  imax r0.y, r0.y, l(0)  
  18:  imin r0.y, r0.x, r0.y  
  19:  imax r0.z, r0.y, r0.z  
  20:  imin r0.x, r0.x, r0.z  
  21:  mov r1.z, l(-1)  
  22:  mov r2.xyz, l(0, 0, 0, 0)  
  23:  loop  
  24:   breakc_nz r2.x  
  25:   ld_structured r0.z, r2.z, l(0), g0.xxxx  
  26:   iadd r3.x, r0.z, r2.y  
  27:   ilt r0.z, r0.y, r3.x  
  28:   iadd r3.y, r2.z, l(1)  
  29:   mov r1.xy, r2.yzyy  
  30:   mov r3.z, r2.x  
  31:   movc r2.xyz, r0.zzzz, r1.zxyz, r3.zxyz  
  32:  endloop  
  33:  mov r0.w, l(-1)  
  34:  mov r1.yz, r2.yyzy  
  35:  mov r1.xw, l(0, 0, 0, 0)  
  36:  loop  
  37:   breakc_nz r1.x  
  38:   ld_structured r2.x, r1.z, l(0), g0.xxxx  
  39:   iadd r1.y, r1.y, r2.x  
  40:   utof r2.x, r2.x  
  41:   utof r2.w, r1.z  
  42:   add r2.w, r2.w, l(0.500000)  
  43:   mul r2.w, r2.w, l(0.011271)  
  44:   exp r2.w, r2.w  
  45:   add r2.w, r2.w, l(-1.000000)  
  46:   mad r3.z, r2.x, r2.w, r1.w  
  47:   ilt r2.x, r0.x, r1.y  
  48:   iadd r2.w, -r2.y, r1.y  
  49:   itof r2.w, r2.w  
  50:   div r0.z, r3.z, r2.w  
  51:   iadd r3.y, r1.z, l(1)  
  52:   mov r0.y, r1.z  
  53:   mov r3.w, r1.x  
  54:   movc r1.xzw, r2.xxxx, r0.wwyz, r3.wwyz  
  55:  endloop  
  56:  store_uav_typed u1.xyzw, l(0, 0, 0, 0), r1.wwww  
  57: endif  
  58: ret  

There is one constant buffer:

Quick look at the assembly: There are two UAVs attached ( u0: input buffer from the first part
and u1: output 1x1 R32_FLOAT texture). We can see that we also have 64 threads per group and 256 elements of 4-bytes groupshared memory.

We start by filling groupshared memory with data from input buffer. We have 64 threads, so we can do it pretty much the same way as before.
To be absolutely sure all data have been loaded for futher processing, we set a barrier after that.

   // The first step is to set whole shared data with data from previous stage.  
   // Because each thread group has 64 threads, each one can fill 4 elements in one thread  
   // using a simple offset.  
   [unroll] for (uint idx=0; idx < 4; idx++)  
   {  
     const uint offset = threadID + idx*64;  
     shared_data[ offset ] = g_buffer[offset];  
   }  
   // We set a barrier here, which means we block execution of all threads in a group until all group   
   // shared accesses have been completed and all threads in the group have reached this call.  
   GroupMemoryBarrierWithGroupSync();  

All calculatons take part in one thread only, all the other ones are used just to load values from buffer to shared memory.
The 'calculating' thread has index of zero. Why? In theory, we could use any thread from [0-63] range, but by comparing with 0, we can avoid extra integer-integer comparison (ieq instruction).

The algorithm is based on specifying range of pixels which will be taken into consideration.
At line 11, we multiply width*height, getting total number of pixels and multiply them by two numbers from [0.0f-1.0f] range which indicate start and end of range. There are some clamps later to make sure that 0 <= Start <= End <= totalPixels - 1:

   // Perform calculations only with the thread with '0' index.  
   [branch] if (threadID == 0)  
   {  
     // Total number of pixels in downscaled buffer  
     uint totalPixels = cb0_v0.x * cb0_v0.y;  

     // Range of pixels (or, more specifically, range of luminance in the screen)
     // we want to incorporate in average luminance calculation.
     int pixelsToConsiderStart = totalPixels * cb0_v0.z;    
     int pixelsToConsiderEnd =  totalPixels * cb0_v0.w;  

     int pixelsMinusOne = totalPixels - 1;  

     pixelsToConsiderStart = clamp( pixelsToConsiderStart, 0, pixelsMinusOne );  
     pixelsToConsiderEnd =  clamp( pixelsToConsiderEnd, pixelsToConsiderStart, pixelsMinusOne );  

As you can see, there are two loops later. The problem with them (or, their assembly) is they have strange conditional moves at the ends. I had a hard time with reconstructing them. Also, take note at line 21. Why is there "-1"? I will reveal it in a few moments.


The purpose of the first loop is to omit pixelsToConsiderStart and give us index of buffer cell in which pixelsToConsiderStart +1 pixel is present (and also number of all pixels in previous cells).

For instance, let's assume that pixelsToConsiderStart is about 30000 and in buffer there are 37000 pixels in cell "zero" (happens during night in the game). So, we want to start analyzing luminance from pixel ~30001, which is present in cell zero. In this scenario, we will exit the loop immediately, having starting index '0' and zero ommitted pixels.

Take a look at the HLSL code:

     // Number of already processed pixels  
     int numProcessedPixels = 0;  
   
     // Luma cell [0-255]  
     int lumaValue = 0;   
   
     // Whether to continue execution of loop  
     bool bExitLoop = false;  
     
     // The purpose of the first loop is to omit "pixelsToConsiderStart" pixels.  
     // We keep number of omitted pixels from previous cells and lumaValue to use in the next loop.  
     [loop]  
     while (!bExitLoop)  
     {  
       // Get number of pixels with specific luma value.  
       uint numPixels = shared_data[lumaValue];  
   
       // Check how many pixels we would have with lumaValue  
       int tempSum = numProcessedPixels + numPixels;  
         
       // If more than pixelsToConsiderStart, exit the loop.  
       // Therefore, we will start calculating luminance from lumaValue.  
       // Simply speaking, pixelsToConsiderStart is number of "darken" pixels to omit before starting calculation.  
       [flatten]  
       if (tempSum > pixelsToConsiderStart)  
       {  
         bExitLoop = true;  
       }  
       else  
       {  
         numProcessedPixels = tempSum;  
         lumaValue++;  
       }  
     }  

This mysterious "-1" from line 21 of the assembly is related with boolean condition of  loop execution (I found it out quite accidentally).

Having the number of pixels from lumaValue cells and lumaValue itself we can go the second loop.
The purpose of the second loop is to calculate contribution of pixels and average luminance.
We start from lumaValue calculated in the first loop.

     float finalAvgLuminance = 0.0f;  
   
     // Number of omitted pixels in the first loop  
     uint numProcessedPixelStart = numProcessedPixels;  
      
     // The purpose of this loop is to calculate contribution of pixels and average luminance.  
     // We start from point calculated in the previous loop, keeping number of omitted pixels and starting lumaValue positon.  
     // We decode luma value from [0-255] range, multiply it by number of pixels which have this specific luma, and sum it up until   
     // we process pixelsToConsiderEnd pixels.   
     // After that, we divide total contribution by number of analyzed pixels.  
     bExitLoop = false;  
     [loop]  
     while (!bExitLoop)  
     {  
       // Get number of pixels with specific luma value.  
       uint numPixels = shared_data[lumaValue];  
         
       // Add to all processed pixels  
       numProcessedPixels += numPixels;  
   
       // Currently processed luma, distributed in [0-255] range (uint)  
       uint encodedLumaUint = lumaValue;  
   
       // Number of pixels with currently processed luma  
       float numberOfPixelsWithCurrentLuma = numPixels;  
   
       // Currently processed, encoded [0-255] luma (float)  
       float encodedLumaFloat = encodedLumaUint;  

At this point we have encoded luma value here [0.0f-255.f].
The decoding process is quite simple - we have to revert calculations from encoding stage.

A quick recap of encoding process:

 float luma = dot( hdrPixelColor, float3(0.2126, 0.7152, 0.0722) ); 
 ...
 float outLuma;          
   
 // because log(0) is undef and log(1) = 0  
 outLuma = luma + 1.0;  
   
 // logarithmically distribute   
 outLuma = log( outLuma );  
   
  // scale by 128, which means log(1) * 128 = 0, log(2,71828) * 128 = 128, log(7,38905) * 128 = 256  
 outLuma = outLuma * 128     
   
 // to uint  
 uint outLumaUint = min( (uint) outLuma, 255);  

To decode luma, we simply revert the encoding process, for example like this:

 // we start by adding 0.5f (we don't want to have zero result)  
 float fDecodedLuma = encodedLumaFloat + 0.5;  
 
 // and decode luminance:
   
 // Divide by 128  
 fDecodedLuma /= 128.0;   
   
 // exp(x) which cancels log(x)         
 fDecodedLuma = exp(fDecodedLuma);  
   
 // Subtract 1.0  
 fDecodedLuma -= 1.0;        

The we calculate contribution by multiplying the number of pixels which have this specific luma times decoded luma, and sum it up until we process pixelsToConsiderEnd pixels.
After that, we divide total contribution by number of analyzed pixels.

See the rest of the loop (and shader):

   // Calculate contribution of this luma  
   float fCurrentLumaContribution = numberOfPixelsWithCurrentLuma * fDecodedLuma;  
         
   // (Temporary) contribution from all previous passes and current one.  
   float tempTotalContribution = fCurrentLumaContribution + finalAvgLuminance;  
     
   
   [flatten]   
   if (numProcessedPixels > pixelsToConsiderEnd )  
   {  
     // to exit the loop  
     bExitLoop = true;  
   
     // We already processed all pixels we wanted, so perform final division here.  
     // Number of all processed pixels from user-selected start  
     int diff = numProcessedPixels - numProcessedPixelStart;  
   
     // Calculate final average luminance  
     finalAvgLuminance = tempTotalContribution / float(diff);  
   }  
   else  
   {      
     // Pass current contribution further and increase lumaValue  
     finalAvgLuminance = tempTotalContribution;   
     lumaValue++;  
   }        
 }  
   
 // Save average luminance  
 g_avgLuminance[uint2(0,0)] = finalAvgLuminance;  

The full shader is here, completely compatible with my HLSLexplorer which was crucial for me to efficiently reconstruct calculating average luminance from The Witcher 3 (well, all the other effects too!).

Phew.... some thoughts at the end. In terms of calculating average luminance, that was difficult shader to reconstruct. Main reasons:
1) strange 'deferred' checks of loop executing, it took me much more time than I initially assumed,
2) Problems with debugging this compute shader with RenderDoc (v. 1.2).
"ld_structured_indexable" operations are not completely supported, while the result of reading from index 0 is fine, all the other ones give zeroes - which makes the loops going to infinity and beyond.

Although I haven't managed to get the same assembly as the original (see screenshot with difference below), I managed to inject this shader with help of RenderDoc into the pipeline and - guess what - the output result was the same! :)

The result of the battle. Left - my shader, right- original assembly.

I hope you enjoyed it,
Thanks for reading.
M.

Reverse engineering the rendering of The Witcher 3, part 7a - average luminance (histogram/distribution)

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".


Welcome,

Calculating average luminance of current frame can be found in virtually any modern video game. Such value is often used later by eye adaptation and tonemapping. Simple approaches include calculating luma to, let's say, 512² texture and calculating its mips and using the last one. This usually works, but is quite limiting. More sophisticated solutions use compute shaders in order to perform, for instance, parallel reduction.

Let's see how this problem was approached by CD Projekt Red in The Witcher 3. I've already investigated its tonemapping and eye adaptation (links in the first paragraph) before and average luminance is the only piece of puzzle missing so far.

To start, calculating average luminance in The Witcher 3 consists of two passes. I decided not to combine them in one post for clarity, so today I will focus on the first one - "distribution of luminance" (calculating histogram of brightness). For the second part, click here to read it.

Finding these two passes shouldn't be too difficult in your favourite frame analyzer. They are subsequent Dispatch calls, just before eye adaptation:

Let's see the inputs for this pass. There are two textures needed:
1) HDR color buffer, downscaled to 1/4 x 1/4 (for example, from 1920x1080 to 480x270),
2) Fullscreen depth buffer

HDR color buffer at 1/4 x 1/4 resolution. Notice nice trick that this buffer is a part of larger one. Reusing buffers is defnitely  a good thing.

Fullscreen depth buffer

Why downscaling color buffer? I guess it's probably all about performance :)

In terms of output for this pass, there is a structured buffer. 256 elements per 4 bytes each.
Shaders have no debug info here, so let's assume it's just a buffer of unsigned ints.

Important: The first step of calculating average luminance is calling ClearUnorderedAccessViewUint to zero all elements of the structured buffer.

Let's see assembly for compute shader (this is the first compute shader in the series!)

 cs_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb0[3], immediateIndexed  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_resource_texture2d (float,float,float,float) t1  
    dcl_uav_structured u0, 4  
    dcl_input vThreadGroupID.x  
    dcl_input vThreadIDInGroup.x  
    dcl_temps 6  
    dcl_tgsm_structured g0, 4, 256  
    dcl_thread_group 64, 1, 1  
   0: store_structured g0.x, vThreadIDInGroup.x, l(0), l(0)  
   1: iadd r0.xyz, vThreadIDInGroup.xxxx, l(64, 128, 192, 0)  
   2: store_structured g0.x, r0.x, l(0), l(0)  
   3: store_structured g0.x, r0.y, l(0), l(0)  
   4: store_structured g0.x, r0.z, l(0), l(0)  
   5: sync_g_t  
   6: ftoi r1.x, cb0[2].z  
   7: mov r2.y, vThreadGroupID.x  
   8: mov r2.zw, l(0, 0, 0, 0)  
   9: mov r3.zw, l(0, 0, 0, 0)  
  10: mov r4.yw, l(0, 0, 0, 0)  
  11: mov r1.y, l(0)  
  12: loop  
  13:  utof r1.z, r1.y  
  14:  ge r1.z, r1.z, cb0[0].x  
  15:  breakc_nz r1.z  
  16:  iadd r2.x, r1.y, vThreadIDInGroup.x  
  17:  utof r1.z, r2.x  
  18:  lt r1.z, r1.z, cb0[0].x  
  19:  if_nz r1.z  
  20:   ld_indexable(texture2d)(float,float,float,float) r5.xyz, r2.xyzw, t0.xyzw  
  21:   dp3 r1.z, r5.xyzx, l(0.212600, 0.715200, 0.072200, 0.000000)  
  22:   imul null, r3.xy, r1.xxxx, r2.xyxx  
  23:   ld_indexable(texture2d)(float,float,float,float) r1.w, r3.xyzw, t1.yzwx  
  24:   eq r1.w, r1.w, cb0[2].w  
  25:   and r1.w, r1.w, cb0[2].y  
  26:   add r2.x, -r1.z, cb0[2].x  
  27:   mad r1.z, r1.w, r2.x, r1.z  
  28:   add r1.z, r1.z, l(1.000000)  
  29:   log r1.z, r1.z  
  30:   mul r1.z, r1.z, l(88.722839)  
  31:   ftou r1.z, r1.z  
  32:   umin r4.x, r1.z, l(255)  
  33:   atomic_iadd g0, r4.xyxx, l(1)  
  34:  endif  
  35:  iadd r1.y, r1.y, l(64)  
  36: endloop  
  37: sync_g_t  
  38: ld_structured r1.x, vThreadIDInGroup.x, l(0), g0.xxxx  
  39: mov r4.z, vThreadIDInGroup.x  
  40: atomic_iadd u0, r4.zwzz, r1.x  
  41: ld_structured r1.x, r0.x, l(0), g0.xxxx  
  42: mov r0.w, l(0)  
  43: atomic_iadd u0, r0.xwxx, r1.x  
  44: ld_structured r0.x, r0.y, l(0), g0.xxxx  
  45: atomic_iadd u0, r0.ywyy, r0.x  
  46: ld_structured r0.x, r0.z, l(0), g0.xxxx  
  47: atomic_iadd u0, r0.zwzz, r0.x  
  48: ret  

And constant buffer:

We know already that the first input is downscaled HDR color buffer. For FullHD, its resolution is 480x270. Take a look at Dispatch call.
Dispatch(270, 1, 1) - that means we run 270 thread groups. Simply speaking, we dispatch one thread group per one row of color buffer.

Each thread group performs on one row of HDR color buffer

Now when we have this context, let's try to figure out what this shader does.
Each thread group has 64 threads in X direction (dcl_thread_group 64, 1, 1) and also some shared memory, 256 elements, 4 bytes per each (dcl_tgsm_structured g0, 4, 256).

Note that in the shader we use SV_GroupThreadID (vThreadIDInGroup.x) [0-63] and SV_GroupID (vThreadGroupID.x) [0-269].

1) We start by setting all elements of shared memory to zero. Since we have 256 elements in shared memory, and 64 threads per group, we can do it nicely with simple loop:

   // The first step is to set whole shared data to zero.  
   // Because each thread group has 64 threads, each one can zero 4 elements using a simple offset.  
   [unroll] for (uint idx=0; idx < 4; idx++)  
   {  
     const uint offset = threadID + idx*64;  
     shared_data[ offset ] = 0;  
   }  

2) After that, we set a barrier with GroupMemoryBarrierWithGroupSync (sync_g_t). We do it to make sure all threads set elements of groupshared memory to zero before going to the next stage.

3) Now we perform loop which we can roughly write like this:

  // cb0_v0.x is width of downscaled color buffer. For 1920x1080, it's 1920/4 = 480;  
   float ViewportSizeX = cb0_v0.x;  
   [loop] for ( uint PositionX = 0; PositionX < ViewportSizeX; PositionX += 64 )  
   {  
      ...  

This is simple 'for' loop with incrementation by 64 (have you already noticed why? ;) ).

The next step it to calculate position of pixel to load.
Let's think about it.

In terms of "Y" coordinate - we can use SV_GroupID.x, because we dispatched 270 thread groups.
In terms of "X" well... we can take advantage of current thread in the group! Let's try it.

Because we have 64 threads per group, such approach will get through all pixels.
Consider thread group (0, 0, 0).
- Thread (0, 0, 0) will process pixels (0, 0), (64, 0), (128, 0), (192, 0), (256, 0), (320, 0),
(384, 0), (448, 0).
- Thread (1, 0, 0) will process pixels (1, 0), (65, 0), (129, 0), (193, 0), (257, 0), (321, 0), (385, 0), (449, 0)
...
- Thread (63, 0, 0) will process pixels (63, 0), (127, 0), (191, 0), (255, 0), (319, 0),
(383, 0), (447, 0)
This way, all pixels will be processed.

We want also to make sure that we won't load pixel out of color buffer:

  // We move along X axis, pixel by pixel. Y is GroupID.  
     uint CurrentPixelPositionX = PositionX + threadID;  
     uint CurrentPixelPositionY = groupID;  
     if ( CurrentPixelPositionX < ViewportSizeX )  
     {  
        // HDR Color buffer.  
        // Calculate screen space position of HDR color buffer, load it and calculate luma.  
        uint2 colorPos = uint2(CurrentPixelPositionX, CurrentPixelPositionY);  
        float3 color = texture0.Load( int3(colorPos, 0) ).rgb;  
        float luma = dot(color, LUMA_RGB);  

See? Pretty simple :)
I've also calculted luma (line 21 of the assembly).

Okay, we already calculated luma from color pixel, feels good. The next step is to load (no samping!) corresponding depth value.
But we have a problem here, because we attached full-resolution depth buffer. How to deal with it?
That's surprisingly simple, just multiply colorPos by some constant (cb0_v2.z). We downscaled HDR color buffer by 4, so this value is 4!

     const int iDepthTextureScale = (int) cb0_v2.z;  
     uint2 depthPos = iDepthTextureScale * colorPos;  
     float depth = texture1.Load( int3(depthPos, 0) ).x;  

So far so good! But... we came to assembly lines 24-25....

  24:   eq r2.x, r2.x, cb0[2].w  
  25:   and r2.x, r2.x, cb0[2].y  

Well. At first, we have floating-poing equality comparison, the result of it goes to r2.x and right after that we have.... what? Bitwise AND?? Seriously? On floating-point value? What the heck???

The 'eq+and' problem
Let me just say this was the most difficult part of this shader to figure out for me. I tried even some crazy asint/asfloat combinations...
What about a bit different approach? Let's just do a simple float-float comparison in HLSL

 float DummyPS() : SV_Target0  
 {  
   float test = (cb0_v0.x == cb0_v0.y);  
   return test;  
 }  

And output assembly:

   0: eq r0.x, cb0[0].y, cb0[0].x  
   1: and o0.x, r0.x, l(0x3f800000)  
   2: ret   

Interesting, isn't it? didn't expect here 'and'.
0x3f800000 is simply 1.0f... well, logical, as we have 1.0 if comparison passes, 0.0 otherwise.
What if you could 'replace' 1.0 with some other value? Like this:

 float DummyPS() : SV_Target0  
 {  
   float test = (cb0_v0.x == cb0_v0.y) ? cb0_v0.z : 0.0;  
   return test;  
 }  

And result:

   0: eq r0.x, cb0[0].y, cb0[0].x  
   1: and o0.x, r0.x, cb0[0].z  
   2: ret   

Hahah! It works :) Just magic by HLSL compiler. Note aside, if you replace 0.0 with something different, it will be just movc.


Going back to our compute shader, the next step is to check if depth value is equal to cb0_v2.w. It's always set to 0.0 - simply speaking, we check if the pixel lies on far plane (sky). If yes, we assign to this factor some value, around 0.5 (I checked few frames).

Such calculated coefficent is used for interpolation between color luma, and 'sky' luma (cb0_v2.x, often around 0.0). I guess this is to give more control how sky is important in calculating average luminance, usually by decreasing its importance. Very smart idea.

    // We check if pixel lies on far plane (sky). If yes, we can specify how it will be  
    // mixed with our values.  
    float value = (depth == cb0_v2.w) ? cb0_v2.y : 0.0;  
         
    // If 'value' is 0.0, this lerp will simply give us 'luma'. However, if 'value' is different  
    // (often around ~0.50), calculated luma can have less importance. (cb0_v2.x is usually close to 0.0).  
    float lumaOk = lerp( luma, cb0_v2.x, value );  
   

As we have lumaOk, the next step is to calculate its natural logarithm to make it distribute nicely. But wait. Let's say that lumaOk is 0.0. We know that log(0) is undefined, so we add 1.0, because log(1) = 0.0.

After that, we scale the calculated logarithm by 128 to distribute it nicely for 256 cells. Very smart!
And this is exactly where this 88.722839 comes from. It's 128 * natural logatithm(2).
It's just the way HLSL calculates logatithms.
In HLSL assembly there is only one function which calculates logarithms: log and it's base-2.

       // Let's assume that lumaOk is 0.0.  
       // log(0) is undefined  
       // log(1) = 0.  
       // calculate natural logarithm of luma  
       lumaOk = log(lumaOk + 1.0);  
         
       // Scale logarithm of luma by 128  
       lumaOk *= 128;  

Finally we calculate index of cell from logarithmically distributed luminance and add '1' to corresponding cell in shared memory.

       // Calculate proper index. Uint and since we have 256 elements in array,  
       // make sure it will not get out of bounds.  
       uint uLuma = (uint) lumaOk;  
       uLuma = min(uLuma, 255);  
   
       // Add '1' to corresponding luma value.  
       InterlockedAdd( shared_data[uLuma], 1 );  

The next step is to, again, set barrier to make sure all pixels in row have been processed.
And the last one is to add values from shared memory to structured buffer, the same way, in simple loop:

   // Wait until all pixels in this row have been processed  
   GroupMemoryBarrierWithGroupSync();  
   
   // Add calculated values to structured buffer.  
   [unroll] for (uint idx = 0; idx < 4; idx++)  
   {  
     const uint offset = threadID + idx*64;  
   
     uint data = shared_data[offset];  
     InterlockedAdd( g_buffer[offset], data );  
   }  

After all 64 threads in the thread group fill shared data, each thread will add 4 values to output buffer.

In terms of the output buffer. Let's think about it. The sum of all values of the buffer is equal to total number of pixels! (for 480x270 = 129 600). So we know now how much of pixels have specific luminance.

If you're a bit rusty with compute shaders (like me) that might not be intuitive at the first time, so get through the post a few times, take pen&paper and try to understand concepts behind this technique.

That's all! :)  That's how The Witcher 3 calculates histogram of luminance. I've certainly learned a lot during writing this post. Congatulations for people at CD Projekt Red!

If you are interested in full HLSL shader, it's here. My ambition always is to get as similar assembly as in original game and I'm more than happy that I've done this again! :)

I hope you enjoyed this post.
Thanks for reading!