Defold Silhouette Demo (work in progress)

[mozokevgen]

I have a demo to share. It works in a simple case. But it has several issues and maybe someone could advise me how to solve them.
But first the working part…

This is an example of how to achieve silhouette effect when sprite are not visible against other sprites. One can use this to highlight a hero in a top-down game, when it is behind buildings or other objects.

Based on this blog post by Valentin Fritz.
You can find source code here: GitHub - mozok/defold-silhouette-demo.

This effect was achieved with usage of custom materials and modified render script.

Materials:

• Player - this is just builtin Sprite material, but with a unique player tag.

• Sprite, Label, Tile Map materials - they have additional discard in the shader when alpha == 0, to prevent transparent parts of the sprite to block the player.

• Silhouette - material just to show solid color. You can change color inside the shader, or send it as a constant. Be aware of additional artifacts if color non-black (0, 0, 0). It could be partially fixed with additional discard (see comment in the shader).

Custom Render script.

Main “magic” happens in custom render script. All changes are made inside the update function.

    ...
    -- render `model` predicate for default 3D material
    --
    render.enable_state(render.STATE_CULL_FACE)
    render.draw(predicates.model, camera_world.frustum)
    -- render.set_depth_mask(false) -- [1]
    render.disable_state(render.STATE_CULL_FACE)

    -- @see https://blog.vfrz.fr/2d-silhouette-effect-in-opengl/#draw-the-silhouette-only-where-needed

    render.enable_state(render.STATE_STENCIL_TEST) -- [2]
    render.set_stencil_func(render.COMPARE_FUNC_ALWAYS, 1, 0xff)
    render.set_stencil_op(render.STENCIL_OP_KEEP, render.STENCIL_OP_KEEP, render.STENCIL_OP_REPLACE)

    render.enable_state(render.STATE_BLEND)
    render.draw(predicates.tile, camera_world.frustum) -- [3]

    render.set_stencil_func(render.COMPARE_FUNC_EQUAL, 1, 0xff)
    render.set_stencil_op(render.STENCIL_OP_KEEP, render.STENCIL_OP_KEEP, render.STENCIL_OP_INCR)
    render.set_color_mask(false, false, false, false)  -- [4]
    render.set_depth_mask(false)
    render.draw(predicates.player, camera_world.frustum)

    render.set_color_mask(true, true, true, true)
    render.set_depth_mask(true)
    render.disable_state(render.STATE_DEPTH_TEST) -- [5]
    render.enable_material("silhouette")
    render.draw(predicates.player, camera_world.frustum)
    render.disable_material("silhouette")

    render.enable_state(render.STATE_DEPTH_TEST)
    render.disable_state(render.STATE_STENCIL_TEST)
    --render.set_depth_mask(false) -- for test with Spine
    render.draw(predicates.player, camera_world.frustum) -- [6]
    --render.set_depth_mask(true) -- for test with Spine

    render.draw(predicates.particle, camera_world.frustum)
    render.disable_state(render.STATE_DEPTH_TEST)

    render.set_depth_mask(false)

    render.draw_debug3d() -- [7]
    ...

1. keep render.set_depth_mask(true) enabled.

2. enable and configure stencil test.

3. draw all tile predicate

4. draw player predicate only to stencil buffer

5. disable STATE_DEPTH_TEST and draw player with silhouette material

6. draw player as usual, with render.set_depth_mask(false)

7. draw everything else (debug, gui) as usual

Example works without an additional stencil mask, but it would have some artifacts.

Demo examples:

• base-example - show simple configuration with 3 sprites on different z-position and a background.

• simple-example - has a tilemap with several objects on it and a hero. Hero move script changes his z-position according to y-position. All objects has their own z-position.

Problems:

• Usage of discard in materials. I’ve heard this is a bad practice. And it leads to additional artifacts (see Label on player object at base demo).

• If the hero “holds” some objects, they are colored with silhouettes too. You need to play with grouping your objects and draw them as separate predicate.

• I can’t get it working with Spine models. It gives me additional artifacts, and I don’t know how to overcome them yet.

• This technique does additional Draw calls. Which could be bad for performance.

• Changing silhouette color to non black leads to additional artifacts (partly fixed).

• Not tested with 3D models.

---

My main issue is Spine models. All this silhouette journey was made to use it with Spine in my game
First tests gives me this effect:

With render.set_depth_mask(true) always enabled it looks like Spine model starts to z-fighting. In my game player and NPCs are Spine models, while other objects are just sprites. I need to figure out how to draw them with right z-positioning to each other and objects around them…

If I enclose the player with render.set_depth_mask(false) ... render.set_depth_mask(true) I still have issue with pickable objects. They should render behind the player on the floor, and before the player when picked up. And silhouette should not affect them.

Any suggestions and advice are welcomed