Reverse engineering the rendering of The Witcher 3, part 6 - sharpen

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".


Hi,

Today we will take a closer look at another postprocess from The Witcher 3 - sharpen.
Sharpening makes an output image a bit crisper. The effect is known from Photoshop and other image editors.

In The Witcher 3 sharpening has two presets: low and high. I will discuss differences between them later, let's take a look at some screenshots now:

"Low" setting - before

"Low" setting - after

"High" setting - before

"High" setting - after

If you want to see more (interactive) comparisons, see section in Nvidia's performance guide for The Witcher 3. As you can see, the effect is particularly visible on grass and foliage.

In this post we will investigate frame from the very beginning of the game: I selected this one purposefully, because here we see terrain (long draw distance) and skydome.

In terms of input, sharpening requires color buffer t0 (LDR, after tonemapping and lens flares) and depth buffer t1.

Let's see the pixel shader, assembly:

 ps_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb3[3], immediateIndexed  
    dcl_constantbuffer cb12[23], immediateIndexed  
    dcl_sampler s0, mode_default  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_resource_texture2d (float,float,float,float) t1  
    dcl_input_ps_siv v0.xy, position  
    dcl_output o0.xyzw  
    dcl_temps 7  
   0: ftoi r0.xy, v0.xyxx  
   1: mov r0.zw, l(0, 0, 0, 0)  
   2: ld_indexable(texture2d)(float,float,float,float) r0.x, r0.xyzw, t1.xyzw  
   3: mad r0.x, r0.x, cb12[22].x, cb12[22].y  
   4: mad r0.y, r0.x, cb12[21].x, cb12[21].y  
   5: max r0.y, r0.y, l(0.000100)  
   6: div r0.y, l(1.000000, 1.000000, 1.000000, 1.000000), r0.y  
   7: mad_sat r0.y, r0.y, cb3[1].z, cb3[1].w  
   8: add r0.z, -cb3[1].x, cb3[1].y  
   9: mad r0.y, r0.y, r0.z, cb3[1].x  
  10: add r0.y, r0.y, l(1.000000)  
  11: ge r0.x, r0.x, l(1.000000)  
  12: movc r0.x, r0.x, l(0), l(1.000000)  
  13: mul r0.z, r0.x, r0.y  
  14: round_z r1.xy, v0.xyxx  
  15: add r1.xy, r1.xyxx, l(0.500000, 0.500000, 0.000000, 0.000000)  
  16: div r1.xy, r1.xyxx, cb3[0].zwzz  
  17: sample_l(texture2d)(float,float,float,float) r2.xyz, r1.xyxx, t0.xyzw, s0, l(0)  
  18: lt r0.z, l(0), r0.z  
  19: if_nz r0.z  
  20:  div r3.xy, l(0.500000, 0.500000, 0.000000, 0.000000), cb3[0].zwzz  
  21:  add r0.zw, r1.xxxy, -r3.xxxy  
  22:  sample_l(texture2d)(float,float,float,float) r4.xyz, r0.zwzz, t0.xyzw, s0, l(0)  
  23:  mov r3.zw, -r3.xxxy  
  24:  add r5.xyzw, r1.xyxy, r3.zyxw  
  25:  sample_l(texture2d)(float,float,float,float) r6.xyz, r5.xyxx, t0.xyzw, s0, l(0)  
  26:  add r4.xyz, r4.xyzx, r6.xyzx  
  27:  sample_l(texture2d)(float,float,float,float) r5.xyz, r5.zwzz, t0.xyzw, s0, l(0)  
  28:  add r4.xyz, r4.xyzx, r5.xyzx  
  29:  add r0.zw, r1.xxxy, r3.xxxy  
  30:  sample_l(texture2d)(float,float,float,float) r1.xyz, r0.zwzz, t0.xyzw, s0, l(0)  
  31:  add r1.xyz, r1.xyzx, r4.xyzx  
  32:  mul r3.xyz, r1.xyzx, l(0.250000, 0.250000, 0.250000, 0.000000)  
  33:  mad r1.xyz, -r1.xyzx, l(0.250000, 0.250000, 0.250000, 0.000000), r2.xyzx  
  34:  max r0.z, abs(r1.z), abs(r1.y)  
  35:  max r0.z, r0.z, abs(r1.x)  
  36:  mad_sat r0.z, r0.z, cb3[2].x, cb3[2].y  
  37:  mad r0.x, r0.y, r0.x, l(-1.000000)  
  38:  mad r0.x, r0.z, r0.x, l(1.000000)  
  39:  dp3 r0.y, l(0.212600, 0.715200, 0.072200, 0.000000), r2.xyzx  
  40:  dp3 r0.z, l(0.212600, 0.715200, 0.072200, 0.000000), r3.xyzx  
  41:  max r0.w, r0.y, l(0.000100)  
  42:  div r1.xyz, r2.xyzx, r0.wwww  
  43:  add r0.y, -r0.z, r0.y  
  44:  mad r0.x, r0.x, r0.y, r0.z  
  45:  max r0.x, r0.x, l(0)  
  46:  mul r2.xyz, r0.xxxx, r1.xyzx  
  47: endif  
  48: mov o0.xyz, r2.xyzx  
  49: mov o0.w, l(1.000000)  
  50: ret  

50 lines of assembly seems like pretty doable task. Let's start it then.

Sharpen amount generation

The first step is to Load depth buffer (line 1). Note that The Witcher 3 uses revesed depth (1.0 - near, 0.0 - far). As you may know, hardware depth is mapped in non-linear way (see this article for details).

Lines 3-6 perform very interesting way of mapping this hardware depth [1.0 - 0.0] to [near-far] values (you set them during MatrixPerspectiveFov). See values from constant buffer:

Having near value of 0.2 and far of 5000 I believe you can calculate values of cb12_v21.xy this way:

cb12_v21.y = 1.0 / near
cb12_v21.x = - (1.0 / near) + (1.0 / near) * (near / far)

This piece of code appears quite often in shaders from TW3, so I believe it's just a function.

When we already have "frustum depth", line 7 uses scale/bias to create a interpolation coefficient (we use saturate here to make sure it's clamped to [0-1] range).

cb3_v1.xy are intensities of sharpening at near and far distances, respectively. Let's call them "sharpenNear" and "sharpenFar". And this is the only difference between "Low" and "High" presets of this effect in The Witcher 3.

Now it's time to use the obtained coefficient. Lines 8-9 are just lerp(sharpenNear, sharpenFar, interpolationCoeff). What is this for? Thanks to that we can have different intensity near Geralt and far away from him). See:

It may be barely visible, but here we interpolated the intensity of sharpen near the player (2.177151) with intensity of the effect far away (1.91303) based on distance. Once we have calculated it we add 1.0 (line 10) to intensity. What is this for? Let's assume that lerp from above gave us 0.0. When we add 1.0 we will have 1.0 of course and this is value which will not affect the pixel during sharpening. More on this later.

During sharpening process we don't want to affect sky. We can achieve this using simple conditional test:

   // Do not perform sharpen on sky  
   float fSkyboxTest = (fDepth >= 1.0) ? 0 : 1;  

In The Witcher 3 depth value for sky pixels is 1.0, so we use it to get some sort of "binary filter" (fun fact: step does not work properly in this case)
Now we can multiply interpolated intenstiy with "sky filter":

This multiplication takes place in line 13.
Example shader code:

   // Calculate final sharpen amount  
   float fSharpenAmount = fSharpenIntensity * fSkyboxTest;  

Sampling center of the pixel

There is an aspect of SV_Position which will be important here: half-pixel offset. It turns out that pixel at top left corner (0, 0) is not (0, 0) in terms of SV_Position.xy, but (0.5, 0.5). Wow!

Here we want to sample in center of the pixel, so take a look at lines 14-16. We can write it in HLSL:

   // Sample the center of the pixel.   
   // Get rid of "half-pixel" offset from SV_Position.xy.  
   float2 uvCenter = trunc( Input.Position.xy );  

   // Add half-pixel to make sure we will sample the center of the pixel  
   uvCenter += float2(0.5, 0.5);  
   uvCenter /= g_Viewport.xy  

And later we sample input color texture from "uvCenter" texcoords. Don't worry, the effect of the sampling will be the same as using "typical" (SV_Position.xy / ViewportSize.xy).

To sharpen or not to sharpen

The decision whether to sharpen or not is based on fSharpenAmount.

   // Get the value of current pixel  
   float3 colorCenter = TexColorBuffer.SampleLevel( samplerLinearClamp, uvCenter, 0 ).rgb;  
     
   // Final result  
   float3 finalColor = colorCenter;  
   
   if ( fSharpenAmount > 0 )  
   {  
     // do the sharpening here...  
   }  
   
  return float4( finalColor, 1 );  
   

Sharpen

It's time to look at the heart of the algorithm.

Basically:
- sample the input color texture four times at the corners of the pixel,
- add the samples and calculate average value,

- calcuate the difference between "center" and "cornerAverage",

- find maximum absolute component of the difference,

- adjust max. abs. component using scale+biasvalues,

- determine amount of the effect using max. abs. component,

- calculate luma of "centerColor" and "averageColor",
- divide the colorCenter by its luma,

- caclulate the new, interpolated luma using amount of the effect,

- multiply the colorCenter by the new luma

Seems like lots of things and it was a challenge for me to understand it, since I've never played with sharpening filters. 

Let's start with sampling pattern. As you can see in the assembly, there are four texture fetches.
It will be best to show it using this image of pixel (Paint level expert):

All fetches in the shader use bilinear sampling (D3D11_FILTER_MIN_MAG_LINEAR_MIP_POINT).

The offset from center to any corner is (±0.5, ±0.5), depending on corner.

See how this can be done in HLSL? Let's see:

    float2 uvCorner;  
    float2 uvOffset = float2( 0.5, 0.5 ) / g_Viewport.xy;  // remember about division!
    
    float3 colorCorners = 0;  
             
    // Top left corner  
    // -0,5, -0.5  
    uvCorner = uvCenter - uvOffset;  
    colorCorners += TexColorBuffer.SampleLevel( samplerLinearClamp, uvCorner, 0 ).rgb;  
   
    // Top right corner  
    // +0.5, -0.5  
    uvCorner = uvCenter + float2(uvOffset.x, -uvOffset.y);  
    colorCorners += TexColorBuffer.SampleLevel( samplerLinearClamp, uvCorner, 0 ).rgb;  
   
    // Bottom left corner  
    // -0.5, +0.5  
    uvCorner = uvCenter + float2(-uvOffset.x, uvOffset.y);  
    colorCorners += TexColorBuffer.SampleLevel( samplerLinearClamp, uvCorner, 0 ).rgb;  
   
    // Bottom right corner  
    // +0.5, +0.5  
    uvCorner = uvCenter + uvOffset;  
    colorCorners += TexColorBuffer.SampleLevel( samplerLinearClamp, uvCorner, 0 ).rgb;  

So now we have all four samples summed in "colorCorners" variable. Let's perform the next steps:

   // Calculate the average of four corners  
   float3 averageColorCorners = colorCorners / 4.0;  
   
   // Calculate the color difference  
   float3 diffColor = colorCenter - averageColorCorners;  
   
   // Find max absolute RGB component of the difference  
   float fDiffColorMaxComponent = max( abs(diffColor.x), max( abs(diffColor.y), abs(diffColor.z) ) );  
   
   // Adjust this factor  
   float fDiffColorMaxComponentScaled = saturate( fDiffColorMaxComponent * sharpenLumScale + sharpenLumBias );  
   
   // Calculate how much pixel will be sharpened.  
   // Note the "1.0" here - this is why we added "1.0" before to fSharpenIntensity.  
   float fPixelSharpenAmount = lerp(1.0, fSharpenAmount, fDiffColorMaxComponentScaled);  
    
   // Calculate luminance of "center" of the pixel and luminance of average value.  
   float lumaCenter = dot( LUMINANCE_RGB, finalColor );  
   float lumaCornersAverage = dot( LUMINANCE_RGB, averageColorCorners );  
       
   // divide "centerColor" by its luma  
   float3 fColorBalanced = colorCenter / max( lumaCenter, 1e-4 );  
     
   // Calc the new luma  
   float fPixelLuminance = lerp(lumaCornersAverage, lumaCenter, fPixelSharpenAmount);  
       
   // Calc the output color  
   finalColor = fColorBalanced * max(fPixelLuminance, 0.0);  
}

return float4(finalColor, 1.0);

The edge detection is done by calculating max. abs. component of the difference. Smart! See its visualization:

Visualization of maximum absolute component of the difference.

Phew. The final HLSL shader is available here. Sorry for quite poor formatting. Feel free to use my HLSLexplorer and play with the code.

I am happy to say that the code above gives exactly the same assembly as in the game! :)

To sum up, The Witcher 3's sharpening shader is very well written (notice that fPixelSharpenAmount is larger than 1.0! that is interesting...). Also, the primary way to modify intensity of the effect are near/far intensities. In the game, they are not constant throughout the gamplay; I collected some example values:

Skellige:

	sharpenNear	sharpenFar	sharpenDistanceScale	sharpenDistanceBias	sharpenLumScale	sharpenLumBias
low	0.40	0.20	0.025	-0.25	-13.33333	1.33333
high	2.0	1.8	0.025	-0.25	-13.33333	1.33333

Kaer Morhen:

	sharpenNear	sharpenFar	sharpenDistanceScale	sharpenDistanceBias	sharpenLumScale	sharpenLumBias
low	0.57751	0.31303	0.06665	-0.33256	-1.0	2.0
high	2.17751	1.91303	0.06665	-0.33256	-1.0	2.0

That's it for today. I hope you enjoyed it :)
Thanks for reading!

M.

Few words about HLSLexplorer

Welcome,

HLSLexplorer started as a hobby project (which still is!) and was never meant to be "Godbolt for shaders" nor any "competition" for already available solutions (see Shader Playground by Tim Jones for example). I just realised during its development that someone might find it useful - that's all.

Today I am happy to say that new version of HLSLexplorer is done and now it's open source!
Let's go through the most important changes.

HLSLexplorer 1.0 in action

Support for modern DirectX compiler and AMD GCN ISA
There are three tabs on the right window. DXBC (d3dcompiler_47.dll for Shader Model 4.0-5.1) and new ones: DXIL (for modern DirectX shader compiler) and AMD GCN ISA

Real-time Pixel Shader Preview
I thought it would be nice to have an option to visualize the result of pixel shader in real time, so I made a tool to do it.

If you click Insert -> Insert simple PS, a dummy pixel shader appear on the left.
Press F7 to start pixel shader preview window:

The left window is rendering output. The right panel is a place to load textures. You can load dds textures, also png/jpg/bmp are supported. Load any texture to texture0 channel... Huh, no result. To see it, switch back to main window, tap F5 to compile shader, and you should see the output in preview window:

Let's go back to dummy pixel shader. There are some differences comparing to previous version of the application:

 cbuffer cbData : register (b12)  
 {  
     float elapsedTime;  
     uint  numFrames;  
     float2 pad;  
   
     float2 viewportSize;  
     float2 viewportInvSize;  
 }  
   
 SamplerState samplerPointClamp  : register (s0);  
 SamplerState samplerPointWrap   : register (s1);  
 SamplerState samplerLinearClamp : register (s2);  
 SamplerState samplerLinearWrap  : register (s3);  
 SamplerState samplerAnisoClamp  : register (s4);  
 SamplerState samplerAnisoWrap   : register (s5);  

Now you have access to some values which are usually available in typical scenarios, like elapsed time (in seconds), viewport / invViewport size and various samplers, you can use them in your pixel shaders and see the effect without closing HLSLexplorer. Pressing F5 updates shader in preview (assuming there is no errors of course). Please note that this feature currently works only with Shader Model 4.0 - 5.0.

HLSLexplorer goes open source
Source code is available on GitHub. If you look just for binary release, click here (Google Drive).

If you would like to build it by yourself:
HLSLexplorer is linked statically against wxWidgets 3.1.1. The compilation of wxWidgets for debug/release configurations should be easy, but please note that I use environmental variable (WXWIN) in HLSLexplorer's solution configuration (additional library directories).

Other improvements worth mentioning
- The program asks if user wants to save HLSL source from the left window when user wants to close the application,
- User can load external hlsl files into the program
- User can save disassembled shaders (DXBC, DXIL, AMD GCN ISA) to hard drive. Just switch to proper window and tap Ctrl+S. The title of saving window tells what you attempt to save.
- Slight UI tweaks
- Improved "About" window ;)

I hope you like it. Go now and disassemble some shaders! :)

Reverse engineering the rendering of The Witcher 3: Index

Welcome,

This is the index page for my "Reverse engineering the rendering of The Witcher 3" small series where I select some rendering techniques from "The Witcher 3", analyze them using RenderDoc, then grab DirectX assembly for shaders and finally try to turn these instructions back to readable HLSL shader.
I focus especially on understanding and explaining presented techniques; it's all for learning purposes after all.

The genesis of the series is simply "I want to know how it's done". Having RenderDoc and other tools which allow us to see every stage of the pipeline is great way to learn how games do rendering stuff.

I have not worked on "The Witcher 3: Wild Hunt" nor its expansion packs in any way so all of code and conclusions here are results of my analysis and tinkering.

Disclaimer: All opinions and views expressed on my blog and on this series are my own and are in no way representative of Rockstar Games.

Here is the current index:

Part 1 - tonemapping

Part 2 - eye adaptation

Part 3 - chromatic aberration

Part 4 - vignette

Part 5 - drunk effect

Part 6 - sharpen

Part 7a - average luminance, part 1 (histogram of luminance)

Part 7b - average luminance, part 2 (calculation)

Part 8 - the Moon and lunar phases

Part 9 - GBuffer

Part 10 - distant rain shafts

Part 11 - lightnings

Part 12 - stupid sky tricks

Part 13a - witcher senses, part 1 (objects & intensity map)

Part 13b - witcher senses, part 2 (outline map)

Part 13c - witcher senses, part 3 (fisheye effect & final combining)

Part 14 - cirrus clouds

Part 15 - fog

Part 16 - shooting stars

Part 17 - the Milky Way

Part 18 - color grading

Part 19 - portals

Part 20 - light shafts

Part 21 - the painted world

Extra:

Someone has done awesome job and prepared Russian version of my posts!
Translation into Russian (parts 1-5)
Translation into Russian (parts 6-10)
Translation into Russian (parts 11-13)

I'm the author of HLSLexplorer which greatly helped me in the process of learning HLSL assembly and was invaluable in writing this series. See the post for details, source code and binaries.


Enjoy! :)

Reverse engineering the rendering of The Witcher 3, part 5 - drunk effect

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".


Hi,

Let's take a look how drunk effect is implemented in The Witcher 3: Wild Hunt.
If you haven't played it yet, drop anything you're doing, buy it and play it see these videos:

Evening:


Night:


At first we see "double rotating" image, pretty common when you're not sober in real life. The more distant the pixel is from the center of image, the rotation effect is stronger. I posted the second video at night on purpose, because you can clearly see this rotation on stars (do you see 8 separate points?)

The second part of TW3 drunk effect, maybe not so visible at first sight, is slight zooming in and out. It's visible near the center.

It's probably obvious that this effect is typical postprocess (pixel shader). However, the order of it in pipeline may not be so obvious. It turns out that drunk effect is applied just *after* tonemapping and just before motion blur (the drunk image is input for motion blur).

Let's start the assembly game:

 ps_5_0  
    dcl_globalFlags refactoringAllowed  
    dcl_constantbuffer cb0[2], immediateIndexed  
    dcl_constantbuffer cb3[3], immediateIndexed  
    dcl_sampler s0, mode_default  
    dcl_resource_texture2d (float,float,float,float) t0  
    dcl_input_ps_siv v1.xy, position  
    dcl_output o0.xyzw  
    dcl_temps 8  
   0: mad r0.x, cb3[0].y, l(-0.100000), l(1.000000)  
   1: mul r0.yz, cb3[1].xxyx, l(0.000000, 0.050000, 0.050000, 0.000000)  
   2: mad r1.xy, v1.xyxx, cb0[1].zwzz, -cb3[2].xyxx  
   3: dp2 r0.w, r1.xyxx, r1.xyxx  
   4: sqrt r1.z, r0.w  
   5: mul r0.w, r0.w, l(10.000000)  
   6: min r0.w, r0.w, l(1.000000)  
   7: mul r0.w, r0.w, cb3[0].y  
   8: mul r2.xyzw, r0.yzyz, r1.zzzz  
   9: mad r2.xyzw, r1.xyxy, r0.xxxx, -r2.xyzw  
  10: mul r3.xy, r0.xxxx, r1.xyxx  
  11: mad r3.xyzw, r0.yzyz, r1.zzzz, r3.xyxy  
  12: add r3.xyzw, r3.xyzw, cb3[2].xyxy  
  13: add r2.xyzw, r2.xyzw, cb3[2].xyxy  
  14: mul r0.x, r0.w, cb3[0].x  
  15: mul r0.x, r0.x, l(5.000000)  
  16: mul r4.xyzw, r0.xxxx, cb3[0].zwzw  
  17: mad r5.xyzw, r4.zwzw, l(1.000000, 0.000000, -1.000000, -0.000000), r2.xyzw  
  18: sample_indexable(texture2d)(float,float,float,float) r6.xyzw, r5.xyxx, t0.xyzw, s0  
  19: sample_indexable(texture2d)(float,float,float,float) r5.xyzw, r5.zwzz, t0.xyzw, s0  
  20: add r5.xyzw, r5.xyzw, r6.xyzw  
  21: mad r6.xyzw, r4.zwzw, l(0.707000, 0.707000, -0.707000, -0.707000), r2.xyzw  
  22: sample_indexable(texture2d)(float,float,float,float) r7.xyzw, r6.xyxx, t0.xyzw, s0  
  23: sample_indexable(texture2d)(float,float,float,float) r6.xyzw, r6.zwzz, t0.xyzw, s0  
  24: add r5.xyzw, r5.xyzw, r7.xyzw  
  25: add r5.xyzw, r6.xyzw, r5.xyzw  
  26: mad r6.xyzw, r4.zwzw, l(0.000000, 1.000000, -0.000000, -1.000000), r2.xyzw  
  27: mad r2.xyzw, r4.xyzw, l(-0.707000, 0.707000, 0.707000, -0.707000), r2.xyzw  
  28: sample_indexable(texture2d)(float,float,float,float) r7.xyzw, r6.xyxx, t0.xyzw, s0  
  29: sample_indexable(texture2d)(float,float,float,float) r6.xyzw, r6.zwzz, t0.xyzw, s0  
  30: add r5.xyzw, r5.xyzw, r7.xyzw  
  31: add r5.xyzw, r6.xyzw, r5.xyzw  
  32: sample_indexable(texture2d)(float,float,float,float) r6.xyzw, r2.xyxx, t0.xyzw, s0  
  33: sample_indexable(texture2d)(float,float,float,float) r2.xyzw, r2.zwzz, t0.xyzw, s0  
  34: add r5.xyzw, r5.xyzw, r6.xyzw  
  35: add r2.xyzw, r2.xyzw, r5.xyzw  
  36: mul r2.xyzw, r2.xyzw, l(0.062500, 0.062500, 0.062500, 0.062500)  
  37: mad r5.xyzw, r4.zwzw, l(1.000000, 0.000000, -1.000000, -0.000000), r3.zwzw  
  38: sample_indexable(texture2d)(float,float,float,float) r6.xyzw, r5.xyxx, t0.xyzw, s0  
  39: sample_indexable(texture2d)(float,float,float,float) r5.xyzw, r5.zwzz, t0.xyzw, s0  
  40: add r5.xyzw, r5.xyzw, r6.xyzw  
  41: mad r6.xyzw, r4.zwzw, l(0.707000, 0.707000, -0.707000, -0.707000), r3.zwzw  
  42: sample_indexable(texture2d)(float,float,float,float) r7.xyzw, r6.xyxx, t0.xyzw, s0  
  43: sample_indexable(texture2d)(float,float,float,float) r6.xyzw, r6.zwzz, t0.xyzw, s0  
  44: add r5.xyzw, r5.xyzw, r7.xyzw  
  45: add r5.xyzw, r6.xyzw, r5.xyzw  
  46: mad r6.xyzw, r4.zwzw, l(0.000000, 1.000000, -0.000000, -1.000000), r3.zwzw  
  47: mad r3.xyzw, r4.xyzw, l(-0.707000, 0.707000, 0.707000, -0.707000), r3.xyzw  
  48: sample_indexable(texture2d)(float,float,float,float) r4.xyzw, r6.xyxx, t0.xyzw, s0  
  49: sample_indexable(texture2d)(float,float,float,float) r6.xyzw, r6.zwzz, t0.xyzw, s0  
  50: add r4.xyzw, r4.xyzw, r5.xyzw  
  51: add r4.xyzw, r6.xyzw, r4.xyzw  
  52: sample_indexable(texture2d)(float,float,float,float) r5.xyzw, r3.xyxx, t0.xyzw, s0  
  53: sample_indexable(texture2d)(float,float,float,float) r3.xyzw, r3.zwzz, t0.xyzw, s0  
  54: add r4.xyzw, r4.xyzw, r5.xyzw  
  55: add r3.xyzw, r3.xyzw, r4.xyzw  
  56: mad r2.xyzw, r3.xyzw, l(0.062500, 0.062500, 0.062500, 0.062500), r2.xyzw  
  57: mul r0.x, cb3[0].y, l(8.000000)  
  58: mul r0.xy, r0.xxxx, cb3[0].zwzz  
  59: mad r0.z, cb3[1].y, l(0.020000), l(1.000000)  
  60: mul r1.zw, r0.zzzz, r1.xxxy  
  61: mad r1.xy, r1.xyxx, r0.zzzz, cb3[2].xyxx  
  62: mad r3.xy, r1.zwzz, r0.xyxx, r1.xyxx  
  63: mul r0.xy, r0.xyxx, r1.zwzz  
  64: mad r0.xy, r0.xyxx, l(2.000000, 2.000000, 0.000000, 0.000000), r1.xyxx  
  65: sample_indexable(texture2d)(float,float,float,float) r1.xyzw, r1.xyxx, t0.xyzw, s0  
  66: sample_indexable(texture2d)(float,float,float,float) r4.xyzw, r0.xyxx, t0.xyzw, s0  
  67: sample_indexable(texture2d)(float,float,float,float) r3.xyzw, r3.xyxx, t0.xyzw, s0  
  68: add r1.xyzw, r1.xyzw, r3.xyzw  
  69: add r1.xyzw, r4.xyzw, r1.xyzw  
  70: mad r2.xyzw, -r1.xyzw, l(0.333333, 0.333333, 0.333333, 0.333333), r2.xyzw  
  71: mul r1.xyzw, r1.xyzw, l(0.333333, 0.333333, 0.333333, 0.333333)  
  72: mul r0.xyzw, r0.wwww, r2.xyzw  
  73: mad o0.xyzw, cb3[0].yyyy, r0.xyzw, r1.xyzw  
  74: ret  

Two separate constant buffers are being used here. Let's check their values:

Few of them are interesting for us:
cb0_v0.x -> elapsed time (seconds)
cb0_v1.xyzw - viewport & inversed viewport size (aka pixel size)

cb3_v0.x - Rotation around pixel, always set to 1.0.
cb3_v0.y - amount of drunk effect. After triggering it, it does not go on full intensity, but rises from 0.0 to 1.0. This is it.
cv3_v1.xy - pixel offsets (more on this later). This is sin/cos pair, so you can use sincos(time) in shader if you want.
cb3_v2.xy - center of effect, usually float2( 0.5, 0.5 ).

What we want to focus on here is to understand how this works instead of blindly rewriting assembly.

We will start from first lines:

 ps_5_0  
   0: mad r0.x, cb3[0].y, l(-0.100000), l(1.000000)  
   1: mul r0.yz, cb3[1].xxyx, l(0.000000, 0.050000, 0.050000, 0.000000)  
   2: mad r1.xy, v1.xyxx, cb0[1].zwzz, -cb3[2].xyxx  
   3: dp2 r0.w, r1.xyxx, r1.xyxx  
   4: sqrt r1.z, r0.w  

The "0" line is something i called "zoom factor", you'll see why in a minute.
Right after that (line 1), we calculate "rotation offsets". It's just input sin/cos pair multiplied by 0.05.

Lines 2-4: At first, we calculate vector from effect center to texture uv. Then we calculate it's squared distance (3) and regular distance (4) (from center to texel)

Zoomed texture coordinates

Let's take at following assembly:

   8: mul r2.xyzw, r0.yzyz, r1.zzzz  
   9: mad r2.xyzw, r1.xyxy, r0.xxxx, -r2.xyzw  
  10: mul r3.xy, r0.xxxx, r1.xyxx  
  11: mad r3.xyzw, r0.yzyz, r1.zzzz, r3.xyxy  
  12: add r3.xyzw, r3.xyzw, cb3[2].xyxy  
  13: add r2.xyzw, r2.xyzw, cb3[2].xyxy 

Since they're packed this way, we can safely analyse only one pair of floats.
For start, r0.yz are "rotation offsets", r1.z is distance from center to texel, r1.xy is vector from center to texel and r0.x is "zoom factor".

To understand it, let zoomFactor = 1.0 for now, so we can write:

   8: mul r2.xyzw, r0.yzyz, r1.zzzz  
   9: mad r2.xyzw, r1.xyxy, r0.xxxx, -r2.xyzw  
  13: add r2.xyzw, r2.xyzw, cb3[2].xyxy 
r2.xy = (texel - center) * zoomFactor - rotationOffsets * distanceFromCenter + center;

  But zoomFactor = 1.0:
  r2.xy = texel - center - rotationOffsets * distanceFromCenter + center;
  r2.xy = texel - rotationOffsets * distanceFromCenter;

Similarly for r3.xy:

  10: mul r3.xy, r0.xxxx, r1.xyxx  
  11: mad r3.xyzw, r0.yzyz, r1.zzzz, r3.xyxy  
  12: add r3.xyzw, r3.xyzw, cb3[2].xyxy  

  r3.xy = rotationOffsets * distanceFromCenter + zoomFactor * (texel - center) + center 

  But zoomFactor = 1.0:
  r3.xy = rotationOffsets * distanceFromCenter + texel - center + center
  r3.xy = texel + rotationOffsets * distanceFromCenter

Sweet. So right now we basically have current TextureUV (texel) +/- rotation offsets, but what about zoomFactor? Take a look at line 0.
Basically, zoomFactor = 1.0 - 0.1 * drunkAmount. For maximum drunkAmount, zoomFactor = 0.9 and calculating zoomed texcoords is now:

  baseTexcoordsA = 0.9 * texel + 0.1 * center + rotationOffsets * distanceFromCenter
  baseTexcoordsB = 0.9 * texel + 0.1 * center - rotationOffsets * distanceFromCenter

Or, maybe more intuitive, it's just linear interpolation between normalized texture coordinates and center by some factor. This is to "zoom in" image. The best way to understand it is to play with it, so here is a link to Shadertoy which shows it in action.

Texcoords offset

The whole piece of assembly:

   2: mad r1.xy, v1.xyxx, cb0[1].zwzz, -cb3[2].xyxx
   3: dp2 r0.w, r1.xyxx, r1.xyxx  
   5: mul r0.w, r0.w, l(10.000000)  
   6: min r0.w, r0.w, l(1.000000)  
   7: mul r0.w, r0.w, cb3[0].y  
  14: mul r0.x, r0.w, cb3[0].x  
  15: mul r0.x, r0.x, l(5.000000)           // texcoords offset intensity
  16: mul r4.xyzw, r0.xxxx, cb3[0].zwzw     // texcoords offset

produces some sort of gradient, let's call it "offset intensity mask". Actually, it produces two. One in "r0.w" (we will use it later) and second, 5 times stronger, in r0.x (line 15). The latter actually serves as multiplier for texel size, so it affects offset strength.

Sampling - rotation part

Next, a series of texture sampling goes on. There are actually 2 series per 8 samplings, one in each "side". In HLSL we can write this this way:

   static const float2 pointsAroundPixel[8] =
    {
        float2(1.0, 0.0),
        float2(-1.0, 0.0),
        float2(0.707,  0.707),
        float2(-0.707, -0.707),
        float2(0.0,  1.0),
        float2(0.0, -1.0),
        float2(-0.707, 0.707),
        float2(0.707, -0.707)
    };

    float4 colorA = 0;
    float4 colorB = 0;

    int i=0;
    [unroll] for (i = 0; i < 8; i++)
    {
        colorA += TexColorBuffer.Sample( samplerLinearClamp, baseTexcoordsA + texcoordsOffset * pointsAroundPixel[i] );
    }
    colorA /= 16.0;

    [unroll] for (i = 0; i < 8; i++)
    {
        colorB += TexColorBuffer.Sample( samplerLinearClamp, baseTexcoordsB + texcoordsOffset * pointsAroundPixel[i] );
    }
    colorB /= 16.0;

    float4 rotationPart = colorA + colorB;

Trick is, we add to baseTexcoordsA/B additional offset lying on unit circle around pixel multiplied by previously mentioned "texcoords offset intensity". The further from center the pixel is, the radius of circle around the pixel is larger - we sample it 8 times, which is well visible on stars. The values of pointsAroundPixel (multiplies of 45 degrees):

from: https://en.wikipedia.org/wiki/Unit_circle

Sampling - zooming in/out part

The second part of drunk effect in The Witcher 3 is zooming "in and out". Let's see assembly responsible for that:

  56: mad r2.xyzw, r3.xyzw, l(0.062500, 0.062500, 0.062500, 0.062500), r2.xyzw  // the rotation part is stored in r2 register

  57: mul r0.x, cb3[0].y, l(8.000000)
  58: mul r0.xy, r0.xxxx, cb3[0].zwzz
  59: mad r0.z, cb3[1].y, l(0.020000), l(1.000000)
  60: mul r1.zw, r0.zzzz, r1.xxxy
  61: mad r1.xy, r1.xyxx, r0.zzzz, cb3[2].xyxx
  62: mad r3.xy, r1.zwzz, r0.xyxx, r1.xyxx
  63: mul r0.xy, r0.xyxx, r1.zwzz
  64: mad r0.xy, r0.xyxx, l(2.000000, 2.000000, 0.000000, 0.000000), r1.xyxx
  65: sample_indexable(texture2d)(float,float,float,float) r1.xyzw, r1.xyxx, t0.xyzw, s0
  66: sample_indexable(texture2d)(float,float,float,float) r4.xyzw, r0.xyxx, t0.xyzw, s0
  67: sample_indexable(texture2d)(float,float,float,float) r3.xyzw, r3.xyxx, t0.xyzw, s0
  68: add r1.xyzw, r1.xyzw, r3.xyzw
  69: add r1.xyzw, r4.xyzw, r1.xyzw

We see that we have three separate texture fetches, so, 3 different texture coordinates. Let's analyse how texcoords for them are calculated. But first, some inputs for this part:

  float  zoomInOutScalePixels = drunkEffectAmount * 8.0; // line 57
  float2 zoomInOutScaleNormalizedScreenCoordinates = zoomInOutScalePixels * texelSize.xy; // line 58
  float  zoomInOutAmplitude = 1.0 + 0.02*cos(time); // line 59
  float2 zoomInOutfromCenterToTexel = zoomInOutAmplitude * fromCenterToTexel; // line 60

Few words about inputs. We calculate offset in texels (e.g. 8.0 * texel size) which is later added to base uv. Amplitude simply oscillates between 0.98 and 1.02 to give "zooming" feeling, like with zoomFactor in rotation part.

Let's start from pair #1, r1.xy (line 61)

  r1.xy = fromCenterToTexel * amplitude + center
  r1.xy = (TextureUV - Center) * amplitude + Center // you can insert here zoomInOutfromCenterToTexel
  r1.xy = TextureUV * amplitude - Center * amplitude + Center
  r1.xy = TextureUV * amplitude + Center * 1.0 - Center * amplitude
  r1.xy = TextureUV * amplitude + Center * (1.0 - amplitude)
  
  r1.xy = lerp( TextureUV, Center, amplitude);
  
  So:
  float2 zoomInOutBaseTextureUV = lerp(TextureUV, Center, amplitude);

Let's check out pair #2, r3.xy (line 62)

  r3.xy = (amplitude * fromCenterToTexel) * zoomInOutScaleNormalizedScreenCoordinates
        + zoomInOutBaseTextureUV

  So:
  float2 zoomInOutAddTextureUV0 = zoomInOutBaseTextureUV
                      + zoomInOutfromCenterToTexel*zoomInOutScaleNormalizedScreenCoordinates;

Let's check out pair #3, r0.xy (lines 63-64)

  r0.xy = zoomInOutScaleNormalizedScreenCoordinates * (amplitude * fromCenterToTexel) * 2.0 + zoomInOutBaseTextureUV

  So:
  float2 zoomInOutAddTextureUV1 = zoomInOutBaseTextureUV
  + 2.0*zoomInOutfromCenterToTexel*zoomInOutScaleNormalizedScreenCoordinates

All the three texture fetches are added together, this results is stored in r1 register. It's worth noticing that this pixel shader uses sampler with "clamp" addressing.

Combining all together

So, right now we have result of rotating in r2 register and added 3 fetches of zooming in r1 register. Let's see the end lines of the assembly:

  70: mad r2.xyzw, -r1.xyzw, l(0.333333, 0.333333, 0.333333, 0.333333), r2.xyzw  
  71: mul r1.xyzw, r1.xyzw, l(0.333333, 0.333333, 0.333333, 0.333333)  
  72: mul r0.xyzw, r0.wwww, r2.xyzw  
  73: mad o0.xyzw, cb3[0].yyyy, r0.xyzw, r1.xyzw  
  74: ret  

For additional inputs: r0.w comes from line 7, it's our intensity mask and cb3[0].y is amount of drunk effect.

Let's fiind out how it works.
Okay, my first approach was "brute-force" way:

  float4 finalColor = intensityMask * (rotationPart - zoomingPart);
  finalColor = drunkIntensity * finalColor + zoomingPart;
  
  return finalColor;

But what the heck, nobody writes shaders this way
I took pen & paper and wrote this formula:

  finalColor = effectAmount * [intensityMask * (rotationPart - zoomPart)] + zoomPart
  finalColor = effectAmount * intensityMask * rotationPart - effectAmount * intensityMask * zoomPart + zooomPart

  - Let t = effectAmount * intensityMask
  - So we have:
  finalColor = t * rotationPart - t * zoomPart + zoomPart
  finalColor = t * rotationPart + zoomPart - t * zoomPart
  finalColor = t * rotationPart + (1.0 - t) * zoomPart
  finalColor = lerp( zoomingPart, rotationPart, t )

  - Finally:
  finalColor = lerp(zoomingPart, rotationPart, intensityMask * drunkIntensity);

Phew! That was quite a detailed post but this is over ;)
Personally I have learned something during writing that one and hopefully you too!

The full HLSL source is here if you are interested. I checked it with my HLSLexplorer and although there is no direct 1-1 relation with original shader, the difference is so small (1 line less) that I can safely assume it's working :)

Let me know if you liked it.
Thanks for reading! :)
M.

Slides from "Witcher Tricks" are available

Hi,

In case you missed it - you can grab slides from my presentation about rendering in The Witcher 3 from here (in Polish and Engish; also with notes).

The presentation covers general rendering pipeline of a single frame and more detailed description of a few techniques, including storing normals in gbuffer, the Sun, blinking stars, tonemapping and vignette.


Mateusz

Reverse engineering the rendering of The Witcher 3, part 4 - vignette

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".



Welcome back! This time we are going to understand and reverse engineer vignette used in The Witcher 3: Wild Hunt.

Vignette

Vignetting is one of the most widespread postprocessing effects used in games. It's popular in photography as well. Subtly darker corners can produce nice looking effect. There are few types of vignetting. For instance, Unreal Engine 4 uses natural one.

But let's back to The Witcher 3. Click here for interactive comparison to see difference between vignette on/off. It's from The Witcher 3 perf guide from NVIDIA.

Screenshot from The Witcher 3 with enabled vignette.

Please notice that the upper left corner (sky) is not as much darkened as the other parts of image.
I will back to this later.

Implementation details

First of all, there is a minor difference in vignette used in the original version of The Witcher 3 (released May 19, 2015) and The Witcher 3: Blood and Wine. In the former, "inverse gradient" is calculated within pixel shader, while in the latter it was precalculated to 256x256 2d texture:

256x256 texture used as "inverse gradient" in Blood & Wine.

I will use shader from Blood & Wine (btw: great game).
Like in most games, vignette in Witcher 3 is calculated in the final postprocess pixel shader. Let's take a look at assembly:

 ...  
  44: log r0.xyz, r0.xyzx  
  45: mul r0.xyz, r0.xyzx, l(0.454545, 0.454545, 0.454545, 0.000000)  
  46: exp r0.xyz, r0.xyzx  
  47: mul r1.xyz, r0.xyzx, cb3[9].xyzx  
  48: sample_indexable(texture2d)(float,float,float,float) r0.w, v1.zwzz, t2.yzwx, s2  
  49: log r2.xyz, r1.xyzx  
  50: mul r2.xyz, r2.xyzx, l(2.200000, 2.200000, 2.200000, 0.000000)  
  51: exp r2.xyz, r2.xyzx  
  52: dp3 r1.w, r2.xyzx, cb3[6].xyzx  
  53: add_sat r1.w, -r1.w, l(1.000000)  
  54: mul r1.w, r1.w, cb3[6].w  
  55: mul_sat r0.w, r0.w, r1.w  
  56: mad r0.xyz, -r0.xyzx, cb3[9].xyzx, cb3[7].xyzx  
  57: mad r0.xyz, r0.wwww, r0.xyzx, r1.xyzx  
 ...  

Interesting! Looks like vignette uses both gamma (line 46) and linear (line 51) spaces to calculate.
At line 48 we sample "inverse gradient" texture.

cb3[9].xyz is not related to vignette. In every tested frame it was set to float3(1.0, 1.0, 1.0) so this is probably final filter used in fade-in / fade-out effects.

There are three main parameters for TW3 vignette:

• Opacity ( cb3[6].w ) - Affects intensity of the vignette. 0 - no vignette, 1 - max vignette. From my observations it looks like in base The Witcher 3 is somewhere around 1.0, while in Blood & Wine it oscillates somewhere 0.15.
• Color ( cb3[7].xyz ) - The great thing about TW3 vignette is possibility to change color of it. It doesn't have to be black, but in practice.. It's usually set as float3( 3.0 / 255.0, 4.0 / 255.0, 5.0 / 255.0 ) and so on - in general multiplies of 0.00392156 = 1.0/255.0
• Weights ( cb3[6].xyz ) - This is very interesting parameter. I've always seen "flat" vignette, like this:

Typical vignette mask

          But using weights (line 52) we can get very interesting results:

TW3 Vignette mask calculated using weights

     

        Weights are close to 1.0. Take a look at frame's constant buffer data from one of frames from Blood&Wine (magic world with rainbow): This is why bright pixels from previously mentioned sky were not really affected by vignette.

The calculated mask is used to interpolate values between image color and vignette's color.

Code

Here is my implementation of TW3 vignette in HLSL.
GammaToLinear = pow(color, 2.2)

 /*  
 // The Witcher 3 vignette.  
 //  
 // Input color is in gamma space  
 // Output color is in gamma space as well.  
 */  
 float3 Vignette_TW3( in float3 gammaColor, in float3 vignetteColor, in float3 vignetteWeights,  
                      in float vignetteOpacity, in Texture2D texVignette, in float2 texUV )  
 {  
      // Calculate vignette amount based on color in *LINEAR* color space and vignette weights.  
      float vignetteWeight = dot( GammaToLinear( gammaColor ), vignetteWeights );  
   
      // We need to keep vignette weight in [0-1] range  
      vignetteWeight = saturate( 1.0 - vignetteWeight );  
   
      // Multiply by opacity  
      vignetteWeight *= vignetteOpacity;  
   
      // Obtain vignette mask (here is texture; you can also calculate your custom mask here)  
      float sampledVignetteMask = texVignette.Sample( samplerLinearClamp, texUV ).x;  
   
      // Final (inversed) vignette mask  
      float finalInvVignetteMask = saturate( vignetteWeight * sampledVignetteMask );  
   
      // final composite in gamma space  
      float3 Color = lerp( gammaColor, vignetteColor, finalInvVignetteMask );
   
      // * uncomment to debug vignette mask:  
      // return 1.0 - finalInvVignetteMask;  
   
      // Return final color  
      return Color;  
 }  

I hope you like it :) Feel free to comment. You can also try my HLSLexplorer which helped me greatly in understanding HLSL assembly and you can also check my previous posts about Witcher 3 rendering techniques.

As always, please take names of variables with grain of salt - TW3 shaders are processed with D3DStripShader so basically I know almost nothing, it's all about guessing. I am also not responsible for any damages done to you hardware due to this shader ;)

Thanks for reading!




Bonus: Calculating gradient

In The Witcher 3 from 2015 inverse gradient is calculated within pixel shader instead of sampling precalculated texture. Let's take a look at the assembly:

  35: add r2.xy, v1.zwzz, l(-0.500000, -0.500000, 0.000000, 0.000000)  
  36: dp2 r1.w, r2.xyxx, r2.xyxx  
  37: sqrt r1.w, r1.w  
  38: mad r1.w, r1.w, l(2.000000), l(-0.550000)  
  39: mul_sat r2.w, r1.w, l(1.219512)  
  40: mul r2.z, r2.w, r2.w  
  41: mul r2.xy, r2.zwzz, r2.zzzz  
  42: dp4 r1.w, l(-0.100000, -0.105000, 1.120000, 0.090000), r2.xyzw  
  43: min r1.w, r1.w, l(0.940000)  

Luckily for us, this is pretty easy. In HLSL this would be something like this:

 float TheWitcher3_2015_Mask( in float2 uv )  
 {  
      float distanceFromCenter = length( uv - float2(0.5, 0.5) );  
   
      float x = distanceFromCenter * 2.0 - 0.55;  
      x = saturate( x * 1.219512 );          // 1.219512 = 100/82  
   
      float x2 = x * x;  
      float x3 = x2 * x;  
      float x4 = x2 * x2;  
   
      float outX = dot( float4(x4, x3, x2, x), float4(-0.10, -0.105, 1.12, 0.09) );  
      outX = min( outX, 0.94 );  
   
      return outX;  
 }  

So we simply calculate distance from center to texel, doing some magic  (multiply, saturate...) with it and then... we calculate polynomial! Awesome.

HLSLexplorer is out!

Hi,

Finally! For the last few months, I've been quite absent.
Now when I have some free time I'm pleased to share with you with first public beta version of HLSLexplorer (internally known as D3DShaderDisassembler) :)

Quick user guide:

In the left window you type some HLSL and in the right you get its assembly once you hit F6.
Most of compilation options are up to you in the lower panel - you can hide/show it with F5.
There is a possibility to save & load presets. To start quickly you can select from "Insert" menu some simple pixel/vertex/compute shader.
You can also load from "File" menu different d3dcompiler dll.

In general, this tool has proved to be really useful for me. I've learned a lot about HLSL assembly and could reverse engineer some shaders from Witcher 3 - check out my older posts if you are interested: tone mapping, eye adaptation, chromatic aberration.

Download binaries from here: http://bit.ly/2nPFofZ
Source code & github repo will be available soon.

Edit: Check out this post for source code and final version.

One more thing: I'm completely aware that in its current state, HLSLexplorer is far from being usable. I will be very thankful for any feedback / feature requests / suggestions / bug reports.
I hope that at least a few of you will like it.

Cheers!