Hey Linus!
As Mikael mentioned, you most likely need to do some changes to your rendering script.
I think there are a couple of ways to do this, one way is to utilise something called “stencil masks”. I did a quick test and created a custom rendering script that does similar to what you are asking.
So essentially I do a game object with a custom material, a material with a special tag I called “mask”. Then I create my own render script (I copied the one in builtins/render/default.render_script) that renders this “mask” tag into a stencil mask before I render the rest of the game.
I added this to my init(self)
function:
self.mask_pred = render.predicate({"mask"})
And these changed the update(self, dt)
function into this:
function update(self)
render.set_depth_mask(true)
render.clear({[render.BUFFER_COLOR_BIT] = self.clear_color, [render.BUFFER_DEPTH_BIT] = 1, [render.BUFFER_STENCIL_BIT] = 0})
render.set_viewport(0, 0, render.get_window_width(), render.get_window_height())
render.set_view(self.view)
render.set_depth_mask(false)
render.disable_state(render.STATE_DEPTH_TEST)
render.set_projection(vmath.matrix4_orthographic(0, render.get_width(), 0, render.get_height(), -1, 1))
-- 1. We need to disable color drawing while rendering the mask
render.set_color_mask(false, false, false, false)
-- 2. Enable stencil test and setup stencil mask parameters
render.enable_state(render.STATE_STENCIL_TEST)
render.set_stencil_func(render.COMPARE_FUNC_ALWAYS, 1, 255)
render.set_stencil_op(render.STENCIL_OP_KEEP, render.STENCIL_OP_KEEP, render.STENCIL_OP_REPLACE)
render.set_stencil_mask(255)
-- 3. Draw the mask
render.draw(self.mask_pred)
-- 4. Update the stencil function to only let pixel pass that are equal to the mask result
render.set_stencil_func(render.COMPARE_FUNC_EQUAL, 1, 255)
-- 5. Re-enable color drawing
render.set_color_mask(true, true, true, true)
-- 6. Continue as rendering usual! :)
render.enable_state(render.STATE_BLEND)
render.set_blend_func(render.BLEND_SRC_ALPHA, render.BLEND_ONE_MINUS_SRC_ALPHA)
render.disable_state(render.STATE_CULL_FACE)
render.draw(self.tile_pred)
render.draw(self.particle_pred)
render.draw_debug3d()
render.set_view(vmath.matrix4())
render.set_projection(vmath.matrix4_orthographic(0, render.get_window_width(), 0, render.get_window_height(), -1, 1))
render.enable_state(render.STATE_STENCIL_TEST)
render.draw(self.gui_pred)
render.draw(self.text_pred)
render.disable_state(render.STATE_STENCIL_TEST)
render.set_depth_mask(false)
render.draw_debug2d()
end
I hope you can use this as some kind of base or just as inspiration!