Render texture filter

[roccosaienz]

I am implementing a simple (post process) distortion shader: the image in render texture A is distorted by the displacement in render texture B. The material has two texture samplers, one for render texture A and one for render texture B. The fragment shader is just the following lines:

vec4 tint_pm = vec4(tint.xyz * tint.w, tint.w);
vec2 dist = (texture2D(tex1, var_texcoord0).xy - vec2(0.5, 0.5)) * dist_scale.xy;
vec4 color = texture2D(tex0, var_texcoord0.xy + dist) * tint_pm;
gl_FragColor = vec4(color);

I have precision issue with this shader and it seems to me that the material is ignoring the sampler filter settings: linear / nearest has no influence in the output.

Any help? thanks!

[jhonny.goransson]

The shader looks fine to me, so I think we need more information to give any advice here. Do you have a repro? Or can invite to the project? Or at least a screenshot?

[Mathias_Westerdahl]

And, what are the expected spaces/ranges of the var_texcoord0, dist and dist_scale?

[roccosaienz]

Thanks to @Mathias_Westerdahl and @jhonny.goransson.

Here are screenshots of
(1) render target A = non distorted image
(2) render target B = displacement image (in the channel red and green)
(3) the result of the shader with input (1) and (2).

As you can see in (3) there are very evident artifacts.

Note that the result in (3) does NOT change if the min/mag filter is LINEAR or NEAREST, and this is somehow surprising to me. I have tried to change both: (a) the filter settings in the material associated with the shader and (b) the filter settings in the options when creating the render targets A and B.

@Mathias_Westerdahl The ranges are as follows:

1. var_texcoord0 varies in [0,1] x [0,1] and are the texcoord of a quad covering the entire screen
2. dist is computed as in the shader where the sampled texture tex1 has values in [0,1]x[0,1]; only the red and green channels are used as displacement in x and y, respectively.
3. dist_scale is used only to compensate the aspect ratio of the render targets A and B and these two have the same resolutions (the same of the window) and can be assumed to be (1,1).

My conclusion is that the engine is using NEAREST as min/mag filter (for render target B, at least) and this explains the artifacts.

I hope this information helps.

Ciao!

[Mathias_Westerdahl]

Yeah, I think you need to submit a repro case since it’s too hard to guess.

[roccosaienz]

@jhonny.goransson @Mathias_Westerdahl I think I have (finally!) prepared a repro case:

displacementRT.zip (20.9 KB)

In the game window you see from left to right: the non-deformed sprite, the deforming displacement map, the deformed sprite. Note that in main.script the displacement map is drawn with an alpha of 0.5 to reduce the displacement. Moreover in the render the constant dist.xy is set to (1.0, -0.1) to compensate the aspect ratio of the window; finally the source sprite in main.collection is flipped horizontally. However all these are just details.

As you see, the displaced image is jagged. And, I am more and more convinced that the engine is sampling the displacement render target with filter NEAREST. This explain mathematically why the lines are jagged. The filter is set to LINEAR both when creating the render targets and in the post process material.

Thanks for your help!

Ciao!

[jhonny.goransson]

No, the engine is setting the samplers correctly. Here’s a capture from renderdoc:

You can see the filters being set on the left, and zooming in on the right image here, the jaggies are likely from the calculations being incorrect.

Here’s fragment shader state with the texture bindings for that pass:

[roccosaienz]

Thank you so much! I see… I was indeed trying to use renderdoc for this. But I use a mac and it is not that simple for me to try on a windows machine.

I don’t understand what is wrong in the shader computation… it is so straightforward.

Any help is highly appreciated!

[roccosaienz]

The Basic Pixel Displacement on shadertoy:

Shadertoy

has exactly the same computation and no artifact… I still don’t understand.

[roccosaienz]

Hey @jhonny.goransson , any idea about this issue? I keep returning to this once a month but it is still a mystery to me…