Normalizing var_texcoord0 in a sprite fragment shader

[aikelb]

Hi there! As the title says, I’ve been trying to normalize var_texcoord0 for sprites. The solution I have in mind is to attach a script to the game object that will feed the shader with the data it needs to normalize it.

This is the fragment shader i’m using for testing:

varying mediump vec4 position;
varying mediump vec2 var_texcoord0;

uniform lowp sampler2D texture_sampler;
uniform mediump vec4 nomalized_sprite_ps;

void main()
{
    vec2 invertedTexCoords = vec2(var_texcoord0.x, 1 - var_texcoord0.y);
    vec2 normalizedTexCoords = (invertedTexCoords - nomalized_sprite_ps.xy) / nomalized_sprite_ps.zw;
    
    gl_FragColor = vec4(normalizedTexCoords.xy,0,1);
}

And this is the script:

    local atlasSize = vmath.vector3(512, 256, 0); --atlas.width, atlas.height
    local sprite_ps = vmath.vector4(128,0, 64,64); --offset.x, offset.y, size.x, size.y
    --TODO: check sprite rotation

    sprite_ps = vmath.vector4(
        sprite_ps.x / atlasSize.x, sprite_ps.y / atlasSize.y,
        sprite_ps.z / atlasSize.x, sprite_ps.w / atlasSize.y)

    go.set("#sprite", "nomalized_sprite_ps", sprite_ps)

And this is the output:

So, the math is working!
Now I just need to figure out how to remove the hardcoded values for the atlas size (512,256), sprite offset (128,0), sprite size (64,64) and also the sprite rotation within the atlas.

JCash pointed me to the resource.get_atlas method, but I couldn’t find the data I need. I can only find the size in the atlas.animations table, which looks like:

    1 = [table: 0x01d9a8d9ebc0]
    {
        "id" = "asteroid1"
        "height" = 32
        "fps" = 30
        "playback" = 0
        "frame_start" = 1
        "frame_end" = 2
        "flip_horizontal" = false
        "width" = 32
        "flip_vertical" = false
    }

So, am I missing anything? Does anyone know how can I get the missing values?

Thanks!

[sergey.lerg]

Hm, I haven’t tried but resource.get_atlas() should return a table with texture , geometries and animations fields. You not getting these?

[aikelb]

Yup! You’re right, but I’m not sure what to do with the output!

--texture
/_generated_72001ba5.texturec
--animation
1 = [table: 0x01fd61349810]
	{
		"frame_end" = 2
		"flip_horizontal" = false
		"height" = 32
		"fps" = 30
		"playback" = 0
		"frame_start" = 1
		"id" = "asteroid1"
		"width" = 32
		"flip_vertical" = false
	}
--geometry
1 = [table: 0x01fd6332e0d0]
	{
		"vertices" = [table: 0x01fd63310010]
		{
			1 = 0
			2 = 32
			3 = 0
			4 = 0
			5 = 32
			6 = 0
			7 = 32
			8 = 32
		}
		"uvs" = [table: 0x01fd633131b0]
		{
			1 = 320
			2 = 96
			3 = 320
			4 = 64
			5 = 352
			6 = 64
			7 = 352
			8 = 96
		}
		"indices" = [table: 0x01fd6331a0d0]
		{
			1 = 0
			2 = 1
			3 = 2
			4 = 0
			5 = 2
			6 = 3
		}
	}

If I can get the data from those values, I don’t know how

[Mathias_Westerdahl]

1. I think the atlas doesn’t currently contain the width/height.
2. the currently returned table is incomplete, cannot be properly used to find the correct geometry.

So, we’d need to fix those, but also we should fix the vertex format in the material to support local uv coords.

[MasterMind]

having access to local uv coords would be awesome. Please give this feature request a thumbs up on github if you can.

Local UV texture space for sprites in an atlas

[britzl]

@Mathias_Westerdahl

I think the atlas doesn’t currently contain the width/height.

Correct, but you can get it from the underlying texture:

	-- sidenote: why didn't we name this resource.get_atlas_info()?
	local ai = resource.get_atlas("/main/main.a.texturesetc")
	local ti = resource.get_texture_info(ai.texture)
	print(ti.width, ti.height)

[jhonny.goransson]

@britzl

	-- sidenote: why didn't we name this resource.get_atlas_info()?

Because _info() would mean that you retrieve meta-data about the atlas, whereas this function returns the same data that is used to create and/or set the atlas data. I think what we actually want is to return the lua equivalent of the protobuf structure for all these types of resource functions, but in the atlas case it’s not really viable because of the complexity of the data format.

[jhonny.goransson]

I think this could be solved by adding a new semantic type to the vertex formats:
Add a center point semantic for custom vertex formats · Issue #8026 · defold/defold · GitHub - you should be able to calculate normalized coordinates with this feature.

[aikelb]

@jhonny.goransson

valent of the protobuf structure for all these ty

Ok! Thanks for the hint @britzl ! I’m using this script now, and seems to be working fine, but I don’t know if the offset values are right. UVs 1 and 4 seem to be working, but I have no idea how to actually read that data ^^Y.

function init(self)
	local current_sprite = go.get("#sprite", "animation")
	local atlas_id = go.get("#sprite", "image")
	local atlas = resource.get_atlas(atlas_id)

	local animation = lume.match(atlas.animations, function(a)
		return hash(a.id) == current_sprite
	end)
	local animation_index = lume.invert(atlas.animations)[animation]
	local geometry = atlas.geometries[animation_index]
	local texture_info = resource.get_texture_info(atlas.texture)

	local atlas_size = { x= texture_info.width, y = texture_info.height}
    local sprite_offset = {x = geometry.uvs[1], y = geometry.uvs[4]}
    local sprite_size = go.get("#sprite", "size")

	local sprite_ps = vmath.vector4(sprite_offset.x, sprite_offset.y, sprite_size.x, sprite_size.y);

	sprite_ps = vmath.vector4(
		sprite_ps.x / atlas_size.x, sprite_ps.y / atlas_size.y,
		sprite_ps.z / atlas_size.x, sprite_ps.w / atlas_size.y)

	go.set("#sprite", "nomalized_sprite_ps", sprite_ps)
end

So the dither I wanted seems to be working, I still need to check with the planet texture. But hey, progress!

Bonus question: does anyone know how can I prevent the pixels flickering when the camera moves? Seems to be working fine if the camera is still

Edit: zoomed in

[Mathias_Westerdahl]

Another idea was to support both “world” and “local” space for the current texture coordinates.
World, being the atlas UVs, and Local , being the local coordinates.

[jhonny.goransson]

Hm yes that could work as well. I think we might want to support both features actually, they serve different purposes

[Mathias_Westerdahl]

For sure, the centerpoint is also very useful.

[britzl]

@aikelb

Bonus question: does anyone know how can I prevent the pixels flickering

What are the pixels? A shader? Sprite(s)?

[aikelb]

A dither shader:

varying mediump vec4 position;
varying mediump vec2 var_texcoord0;

uniform lowp sampler2D texture_sampler;
uniform lowp vec4 tint;
uniform mediump vec4 nomalized_sprite_ps;
uniform mediump vec4 alpha;

float DitherPlastic(vec2 pos)
{
    return 2.0 * abs(fract(dot(pos, vec2(0.75487767, 0.56984029))) - 0.5);
}

void main()
{
	// Pre-multiply alpha since all runtime textures already are
	lowp vec4 tint_pm = vec4(tint.xyz * tint.w, tint.w);
	

    // Normalize var_texcoord0 using offset and scale
	vec2 invertedTexCoords = vec2(var_texcoord0.x, 1 - var_texcoord0.y);
    vec2 normalizedTexCoords = (invertedTexCoords - nomalized_sprite_ps.xy) / nomalized_sprite_ps.zw;
	vec2 normalizedPixel = floor(normalizedTexCoords * vec2(640,320));

	gl_FragColor = vec4(step(alpha.x, DitherPlastic(normalizedPixel)), 0,0,1);
}

I tried using the floor when normalizing the coords, but still have the same issue. Not sure if it can be fixed. I want to use that value as alpha.

[aikelb]

Just in case anyone needs it, this is the final code I’m using:

function init(self)
	local current_sprite = go.get("#sprite", "animation")
	local atlas_id = go.get("#sprite", "image")
	local atlas = resource.get_atlas(atlas_id)

	local animation = lume.match(atlas.animations, function(a)
		return hash(a.id) == current_sprite
	end)
	local animation_index = lume.invert(atlas.animations)[animation]
	local geometry = atlas.geometries[animation_index]
	local texture_info = resource.get_texture_info(atlas.texture)

	local atlas_size = { x = texture_info.width, y = texture_info.height }
	local sprite_offset = { x = geometry.uvs[1], y = geometry.uvs[4] }
	local sprite_size = go.get("#sprite", "size")

	local sprite_ps = vmath.vector4(sprite_offset.x, sprite_offset.y, sprite_size.x, sprite_size.y)

	sprite_ps = vmath.vector4(
		sprite_ps.x / atlas_size.x,
		sprite_ps.y / atlas_size.y,
		sprite_ps.z / atlas_size.x,
		sprite_ps.w / atlas_size.y
	)

	go.set("#sprite", "nomalized_sprite_ps", sprite_ps)
end

function update(self, dt)
	-- go.set("#sprite", "alpha",  value)
end

The ditther shader, based on Shader - Shadertoy BETA

varying mediump vec4 position;
varying mediump vec2 var_texcoord0;

uniform lowp sampler2D texture_sampler;
uniform lowp vec4 tint;
uniform mediump vec4 nomalized_sprite_ps;
uniform mediump vec4 alpha;

float DitherPlastic(vec2 pos)
{
    return 2.0 * abs(fract(dot(pos, vec2(0.75487767, 0.56984029))) - 0.5);
}

void main()
{
	vec2 res = vec2(640, 360);
	float pixelSize = 48.0 / res.y;

	vec2 invertedTexCoords = vec2(var_texcoord0.x * pixelSize, 1 - var_texcoord0.y * pixelSize);
    vec2 normalizedTexCoords = (invertedTexCoords - nomalized_sprite_ps.xy) / nomalized_sprite_ps.zw;
	vec2 normalizedPixel = floor(normalizedTexCoords * res);

	float a = step(alpha.x, DitherPlastic(normalizedPixel));
  if (a < 0.01) discard;
	lowp vec4 tint_pm = vec4(tint.xyz * tint.w * a, tint.w * a);

	gl_FragColor = texture2D(texture_sampler, var_texcoord0.xy) * tint_pm;
}

And this is how it’s looking

Not bad at all!

[Mathias_Westerdahl]

That’s a nice effect!