Reverse engineering the rendering of The Witcher 3, part 21 - the painted world

This post is a part of the series "Reverse engineering the rendering of The Witcher 3".

If you have played "Hearts of Stone" expansion pack you might recall one of quests when Geralt finds himself in so-called "painted world". What is special about it is it looks much different than the "normal" world.

The following two screenshots present the game before and after this effect is applied. This quest is the only opportunity to see this effect in the game.

As you can see, the effect applied makes the normally rendered scene look like a painting, I get some The Starry Night vibes from it.

Let's find out how this has been implemented.

It's just one fullscreen postprocessing pass, applied near the end of postfx chain. 

I posted the assembly (418 lines!) here. In the post I'll focus on the general idea and will omit some game-specific nuances (for instance, the game performs stencil test to exclude characters from the final effect).

Taking a look at the beginning of the assembly, there is a number of snippets similar to this one:

  12: frc r3.x, r3.x  
  13: mov r1.w, r0.w  
  14: dp2 r0.z, r0.zwzz, l(25.979601, 156.466003, 0.000000, 0.000000)  
  15: sincos r0.z, null, r0.z  
  16: mul r0.z, r0.z, l(43758.546875)  

This indicates the use of one of those 'random' shader functions, you've probably seen this one (or its variants) already:

 float random(in float2 st)   
 {  
    return frac(sin(dot(st.xy, 2.0*float2(12.9898,78.233))) * 43758.5453123);  
 }  

The shader starts with these lines:

   0: mul r0.xy, v0.xyxx, cb12[23].zwzz  
   1: mul r0.y, r0.y, cb12[23].y  
   2: div r0.y, r0.y, cb12[23].x  
   3: mul r0.z, r0.y, l(50.000000)  
   4: round_ni r1.z, r0.z  
   5: mul r0.z, r0.x, l(50.000000)  
   6: round_ni r2.x, r0.z  
   7: mov r1.w, r2.x  

which we can write in HLSL as so:

   static const float TILES = 50;

   float2 pixelPos = Input.PositionH.xy; // SV_Position  
   float2 screenSize = viewportSize.xy;  
   float2 pixelSize = viewportInvSize;  
   
   float2 Texcoords = pixelPos * pixelSize;  
   
   float invAspectRatio = (Texcoords.y * screenSize.y) / screenSize.x;  
     
   // Divide the screen into tiles  
   float2 st;  
   st.x = floor(Texcoords.x * TILES);  
   st.y = floor(invAspectRatio * TILES);  

Using the standard postfx inputs (fullscreen res, pixel pos, pixel size) we divide the screen into a number of 'tiles'. On X axis you have totally 50 of them, on Y 28 (for 16:9 aspect). They are integers so they go like 1, 2, 3, ..., 50 for X axis and similarly for Y one.

Note that most pixels share the same 'tile' value.

Once the 'tile' values are obtained, they serve as inputs to random function. Calling

 float noise1 = random(st);  
   
 return noise1.xxxx;  

gives:

which visualizes tiles nicely.

However, the shader does not call random just once. There are 16 calls to it:

   // get noise  
   float noise1 = random(st);  
   float noise2 = random(st + float2(0, 2));  
   float noise3 = random(st + float2(-1, -1));  
   float noise4 = random(st + float2(2, 2));  
   float noise5 = random(st + float2(2, 0));  
   float noise6 = random(st + float2(2, -1));  
   float noise7 = random(st + float2(-1, 0));  
   float noise8 = random(st + float2(1, 0));  
   float noise9 = random(st + float2(1, 2));  
   float noise10 = random(st + float2(-1, 2));  
   float noise11 = random(st + float2(2, 1));  
   float noise12 = random(st + float2(0, 1));  
   float noise13 = random(st + float2(-1, 1));  
   float noise14 = random(st + float2(0, -1));  
   float noise15 = random(st + float2(1, 1));  
   float noise16 = random(st + float2(1, -1));  

And a small diagram which visualizes the order of the calls:

The shader performs now a number of calculations to get relatively smooth noise (not interested with blocky appeareance!). 

First of all, for each of center 'blocks' (1, 8, 12, 15) we assign weights of 0.25 and we do the similar for their neighbours. It'll be more clear once visualized.

Let's start from the block nr 1:

for the center block (1) we assign a weight of 0.25 (1/4). For the red ones - a weight of 0.125 (1/8) and for the blue ones - a weight of 0.0625 (1/16)

In the shader this can be implemented like so:

   float p1 = 0.0;  
   p1 += (noise3 + noise16 + noise13 + noise15) / 16.0;  
   p1 += (noise14 + noise12 + noise7 + noise8) / 8.0;  
   p1 += noise1 / 4.0;  

For block nr 8 it's the same idea:

   float p8 = 0.0;  
   p8 += (noise14 + noise6 + noise12 + noise11) / 16.0;  
   p8 += (noise16 + noise15 + noise1 + noise5) / 8.0;  
   p8 += noise8 / 4.0;  

And the same idea goes for blocks 12 and 15.

Here's how p1 looks like: 

Much smoother! Yet it's still blocky because most of pixels share the same 'tiles' values (float2 st).

To remove blockiness, the shader performs a number of interpolations. Ideally we would like to have values in [0-1] range within each tile. These can be calculated with:

   // Get frac parts for interpolation  
   float interp_x = TILES*Texcoords.x - st.x;  
   float interp_y = TILES*invAspectRatio - st.y;  

Here's interp_x:

and interp_y:

Having the values, interpolation goes as:

   // First, interpolate between the 'lower' ones: 12 and 15  
   float t_lower = lerp(p12, p15, interp_x);  
  
   // Then interpolate between the 'upper' ones: 1 and 8  
   float t_upper = lerp(p1, p8, interp_x);  
     
   // final interpolate  
   float t = lerp(t_upper, t_lower, interp_y);  

This gives us:

To get circular patterns on the screen, some basic trigonometry is applied:

   t *= 6.283185;  // ~2*PI, full phase
     
   float2 offsets;  
   offsets.x = sin(t);  
   offsets.y = cos(t);  

And here is a visualization of offsets (applied * 0.5 + 0.5 to squish them from [-1;1] to [0;1] range )

Having the offsets we can perform texture sampling (the game uses point clamp sampler)

   float2 minUv = float2(0.5, 0.5) * pixelSize;  
   float2 maxUv = (viewportSize.xy - float2(0.5, 0.5)) * pixelSize;  
   
   float3 color = 0.0.xxx;  
   [unroll] for (int i = -5; i <= 5; i++)  
   {  
     float2 Tex = Texcoords + i * pixelSize*offsets;  
     Tex = clamp(Tex, minUv, maxUv); 
       
     color += texture0.Sample( samplerPointClamp, Tex ).rgb;  
   }  
   color /= float(11);  
   
   return float4(color, 1.0);

And that's pretty much the idea. The code above is, of course, a simplified version. Again, the game does a little bit more (using depth buffer to weaken the effect, using stencil buffer to exclude characters etc.)

I put the shader on Shadertoy if you want to experiment with it.