[SOLVED] How to mask/clipping a GUI?

Hello everyone :wave:

I really need help to create a heart mask. Should work just like a stencil, but in the shape of a heart:


My Project.zip (37.4 KB)

I have been looking for a way to do this for month, through shaders, through rendering, but unfortunately, my knowledge and skills are not enough to achieve at least some result :sob:

I really already regret my choice to make a game on Defold, because it is very difficult for beginners to create something of their own almost from scratch, but we have been making the game with a small team for 5 months and there is no way back, I understand that this is impudent on my part , but if you have an example of how you implemented it, please share :pray:

If you can use game object & sprite instead of GUI nodes, this example could help

I feel they are only focusing on adapting new things for game objects so canā€™t expect GUI will be easier and more convenient when working with it.

Thanks for the GO example :slightly_smiling_face:
It turns out that there are currently no technical possibilities to implement the same for the GUI?

Does it have to be stencil buffer based? Ie does it have to work with gui layers? Otherwise achieving it with texture masks shouldnā€™t be that difficult if you do a bit of custom render script setup

I donā€™t quite understand the question, but I have many GUI layers in my project. We are making a very simple novel game and we use GUI for everything, even for the background, which is why it is impossible to render the GO, because the GUI is rendered last, but I think in the render, draw the GUI first, then the GO should show

Your post mentioned ā€œstencilā€, which is how the GUI layering system is built. It would be more technical to solve your masking if you need it to work exactly like a layer does in the GUI. Implementing the masking part of the look is easier as you can use a texture to do the actual masking. Would that solve your issue?

Instead of a single texture, its show the whole atlas and also in black and white.
dmengine_HX9lDvsTCp

Itā€™s all fixable, but I need more than just a texture, I need to show complex animation through the heart window (stencil), so I give up. My issue can be solved only by the knowledge and skills of shaders and rendering. I think I just realize what I want with frame-by-frame animationā€¦

And all of this needs to be in gui, or would it work with a go solution? Letā€™s see if we canā€™t help you with it, would be a good example I think

1 Like

Iā€™ve just updated my above example to add GUI mask demonstration. Although itā€™s quite tangled, I hope you can find something useful.

1 Like

Yes, something you could make with Photoshop as Iā€™m also using for my project but it quite costs

Oh, thank you so much, can I just ask why he makes the picture black and white? I added pink and green, but they turned out white

This is alpha masking technique, so the mask is always in black and white. The white will make the main texture visible while black will make it hidden.

I may know what you are looking for. Actually you can modify a bit to get what you want

Yes, I got it, thank you very much :smiling_face_with_three_hearts:, that should solve my problem

4 Likes

Thanks everyone, I want to share the result I wanted to achieve. I would also like to make different speeds for the waves, but it is quite difficult, because the mask has coordinates relative to the texture to which it is attached, I tried to add a separate coordinate of the var_texcoord1 for the mask through the shader, but as with the Positions, this is technically not implemented in the GUI or I didnā€™t understand why it didnā€™t work.
Thanks again for your help! :blush:
dmengine_sUrZY4JSnI

5 Likes

To archive the result as you wanted, if you still use the code in my example, you can change a little bit at fragment shader program:
From:

vec4 tex = texture2D(texture_sampler, var_texcoord0.xy);
vec4 masky = texture2D(mask_texture, var_texcoord0.xy - mask_position.xy);

To:

vec4 tex = texture2D(texture_sampler, var_texcoord0.xy - mask_position.xy);
vec4 masky = texture2D(mask_texture, var_texcoord0.xy);

I did as you said but it gave this result GIF, now no matter how i change the mask_position nothing changes.
dmengine_jIKvWr70um

My goal is for the mask to always stay in one position on the screen and the waves to move up and down and slightly left and right.

For now, I implemented the moving heart mask down, and the waves up with the same speed, this way I make the ā€œstanding effectā€ of the mask in a static position:

function update(self, dt)
local speed_y = 5
local speed_x = 10
self.mask_postion.x = self.mask_postion.x - 1/speed_x * dt
self.mask_postion.y = self.mask_postion.y - 1/speed_y * dt -- move down
msg.post("@render:", "set_mask_position", { position = self.mask_postion })

self.current_pos.x = self.current_pos.x + 128/speed_x * dt
self.current_pos.y = self.current_pos.y + 128/speed_y * dt -- move up
if self.current_pos.y >= self.origin_pos.y - 30 then
	self.current_pos.x = self.origin_pos.x
	self.current_pos.y = self.origin_pos.y - 128
	self.mask_postion.x = 0.35
	self.mask_postion.y = 1
end
gui.set_position(self.wave_1, self.current_pos)

end

You do not set position of the node. After set the fragment sharder as in my above, removing the code bellow the line msg.post("@render:", "set_mask_position", ...) might help.

Yes, that helped, managed to achieve the same result, but without the complex calculations, thank you very much for your help :blush::heart:
dmengine_M0csmNiGC4

4 Likes

I need help again :pensive: Iā€™m trying to implement different movement logic for two masks, but only the last wave_2 works, more precisely, wave_1 also works, but according to the movement logic of wave_2. I understand why this happens, but I donā€™t understand how to fix it :disappointed_relieved:

dmengine_GYyB4W26NH

-- render GUI
--
local camera_gui = state.cameras.camera_gui
render.set_view(camera_gui.view)
render.set_projection(camera_gui.proj)
render.enable_state(render.STATE_STENCIL_TEST)

if self.maskable_nodes["wave_1"].mask_sampler then
    if self.maskable_nodes["wave_1"].mask_position then
        if not camera_gui.frustum.constants then camera_gui.frustum.constants = render.constant_buffer() end
        camera_gui.frustum.constants.mask_position = self.maskable_nodes["wave_1"].mask_position
    end
    render.enable_texture("mask_sampler", self.maskable_nodes["wave_1"].mask_sampler)
end
render.draw(predicates.gui, camera_gui.frustum)
if self.maskable_nodes["wave_1"].mask_sampler then
    render.disable_texture("mask_sampler")
end

if self.maskable_nodes["wave_2"].mask_sampler then
    if self.maskable_nodes["wave_2"].mask_position then
        if not camera_gui.frustum.constants then camera_gui.frustum.constants = render.constant_buffer() end
        camera_gui.frustum.constants.mask_position = self.maskable_nodes["wave_2"].mask_position
    end
    render.enable_texture("mask_sampler", self.maskable_nodes["wave_2"].mask_sampler)
end
render.draw(predicates.gui, camera_gui.frustum)
if self.maskable_nodes["wave_2"].mask_sampler then
    render.disable_texture("mask_sampler")
end

render.draw(predicates.debug_text, camera_gui.frustum)
render.disable_state(render.STATE_STENCIL_TEST)
render.disable_state(render.STATE_BLEND)

Thatā€™s why Iā€™m not happy with this solution. You can just use it very limited.

If you want to use it for 2 different nodes in the same GUI scene then I think you can clone the material to a new one with different constant name and mask texture name. And in render script you do not double render.draw(predicates.gui...) line, just double others (with different names)

Hope you understand what Iā€™m saying :grinning_face_with_smiling_eyes:

Also consider to use frame-by-frame animation if all you need is just that heart. It will be more clearly and easy to work with

2 Likes