Can I define my own depth sorting for the sprites when the perspective camera is used?

Questions
#1

Hello everyone!
Before I’m gonna describe my problem I would like to say that I’ve been trying Defold engine for a couple of months already and I really enjoy it. Great job Defold Team!

On my journey I’ve got the following problem, which I’ve been trying to resolve for about a week already and failed. Perhaps there is a solution and I overlooked it but browsing the forum didn’t help me find the answer to my problem, so here we go.

I want to make something similar to pong but with a perspective view from one of the players and it always looks towards the opposite player. I want to use exclusively sprites for the balls, and for the players. That’s not my picture, but the idea should look somewhat similar
image

The whole space is 3D and it has to utilise 3d physics as well. No problems so far and everything sounds achievable without any problem.

The table
This one can be a sprite with a material that has a different predicate that is drawn separately before everything (so it always stays at the background, sort of…)

The Paddles and The Balls
HERE I HAVE THE FOLLOWING ISSUE:
Since the camera is in perspective mode, the depth test is not done by considering Z coordinates, but rather a distance from the camera and that creates some artifacts - specifically, some sprites behind the transparent parts simply don’t get drawn. Here’s an example (not related to Pong but rather created to demonstrate the issue):
The Purple Sprite is in front of the Red Sprite

image
image730×622 55.1 KB

Oops, a little bit of rotation to the right and it gets wrong
image
image807×718 42.1 KB

Of course, that’s predictable behaviour, because the pivot of the red sprite got closer to the camera, and therefore put the last in the Z-Sorting, that’s why it got drawn the last and covered the purple sprite. None of the the changes in the render_script provided good result (because depth test works fine, although it’s not applicable to transparent or semi/transparent objects as I have in the pictures above). Ok-ok, it actually worked sometimes… until I decide to rotate the camera again, and we’re back to the step zero :frowning:

Then I decided to perform the sorting from my custom material by making some changes to the default sprite material (specifcally the vertex shader in it):

    vec4 pos = view_proj * vec4(position.xyz, 1.0);
    pos.z = -position.z * 0.01 + 0.5;
    var_texcoord0 = texcoord0;

    gl_Position = pos;

In these changes I added pos.z = var_depth * 0.01 + 0.5;, where the screen z-coordinate of the vertex is adjusted according to the original object Z-coordinate value (*0.001 + 0.5 is needed to let all objects between -500 to 500 being presented in the range [0; 1] of the screen space, and therefore being drawn)

Unfortunately, this also works just fine until… I decide to rotate the camera… :frowning:

Drawing into a DepthBuffer…somehow first and then drawing again but based on the generated DepthBuffer? Not an option! The problem of depth testing for semi-transparent objects is not generally resolved, and I see why: if the object is half transparent then should it write the depth values into the depth buffer?

image
image673×662 66.2 KB

→ The answer is Yes? Nice, then the objects behind will not be rendered properly since they must be visible to some degree through the front semi-transparent object. You get this:

image
image725×601 64.2 KB

→ The answer is No? Cool, then the transparent objects will be drawn as they come sorted, and that order may not satisfy the needs. Here’s the outcome:

image
image705×643 51.2 KB

Why do I actually need to resolve that?
On the pong table will be multiple balls coming back and forth at different positions in 3d. The balls are square textures with a ball image on it but transparent corners (Yeah, sprites, but 3d colliders):
Example:
image

After all I was surprised that in Defold there is no way to control the way how the predicates are sorted (or do I need to write my own C++ extension? I eat c++ for breakfast so I’m not scared, just wondering :stuck_out_tongue: ), however the render pipeline is pretty well customisable!

All what I need to be happy is simply have the sprites ordered by Z-axis exclusively, regardless of how my camera is oriented or any other variables and parameters.

OR, maybe the vertex shader can take the full responsibility as I've shown above. That will make the overall process much easier that developing some obscure workaround.



Anyway, I hope there is some reasonable solution for that problem. Cheers!
2 Likes
#2

Do you have a working prototype available to make a .gif animation of the problem? I’m not 100% understanding the issue.

#3

This is partly solved in the fragment shader by calling discard for transparent pixels. But this does not work well for opaque surfaces (like glass).

    vec4 color = texture2D(tex0, var_texcoord0.xy) * tint_pm;
    if(color.a < 0.3) discard;
#4

Of course! I don’t have that much time today to do so, but I’ll record it as soon as I can!

#5

I’ve already seen a similar solution on the forum when I was researching the ways to manipulate the depth buffer.

Nevertheless, I don’t appreciate using [if] or [discard] operations in a shader, especially considering that this is supposed to be a mobile game. (For a few objects that will work, but my intention is to add hundreds of sprites if not a couple of thousands)

But this does not work well for opaque surfaces (like glass).

Yes, I wish to incorporate opaque surfaces later, therefore I would like to see if there is a more general solution.

Just as a small recap of my intention: I wish the predicate is sorted not based on the perspective camera view but a specified direction (just like in Orthographic camera). Then the sprites will be drawn exactly as they come sorted from -Z to +Z and no further tricks are needed since my game logic guarantees that it’s the correct order to draw the sprites. Simple and straightforward.

I know that there is no way I can sort them by calling a function, setting up a flag or manipulating some parameters, but is there any other way to achieve it?

#6

There’s a discussion on the topic here, but I don’t know if there’s any progress on a solution.

#7

I have already seen this post. It looks like a feature request, and it hasn’t been fulfilled yet, alas :frowning:

#8

Maybe it just needs to be tested on target devices. For example, I recently ran an example with hundreds of models using discard in the shader on a mid-priced 2017 phone and it worked quite well. Depending on the graphics and the required sharpness of the image for optimisation, sometimes it is more efficient to reduce the render resolution and render to a texture and then upscale it to the screen than to load a low-performance device with calculations for a retina screen.

As an example with discard, if, for loops and double rendering because of shadows you can look at https://github.com/Dragosha/defold-light-and-shadows (online example: Light and Shadows 0.3).

#9

Nice demo, btw! I had played it even before creating this post (yeah, I’ve searched a lot :sweat_smile:)
But you don’t have any opaque sprites there, which definitely simplifies your case. My examples have probably misguided my intention to make it work universally for any sprite: solid, opaque, semi- or fully-transparent.

However, I see that you use a particle system for the candles, right? It’s actually the best showcase of the problem I’m describing.

image
image515×642 46.4 KB

If you could have organised the depth sorting by z-axis, you wouldn’t have needed to create so many different materials used and to have more than a single predicate for all your sprites (except the ground perhaps).

1 Like
#10

Another the best showcase in dynamic :sweat_smile:: https://user-images.githubusercontent.com/842807/177780855-04a7f9d3-ef1e-43e3-8a33-e5280e4cddaf.mp4

(Pivot for sprites (DEF-651) · Issue #3666 · defold/defold · GitHub)

1 Like
#11

I don’t have that problem using a mesh/model component, with a billboard vertex program and discard for alpha scissoring. Correct blending may not be possible, but you can arrange quads in their correct order. It would be a strange 3D engine bug if you couldn’t. :slightly_smiling_face:

#12

Oh, I wish to use quads or some mesh/models for that, but they don’t provide as a simple setup and image animation as sprites do. For example, if I have an atlas of sprites with character animation frames, how would you make that animation running on quads? And if I have multiple animations? Yeah, it gets more complicated than it should be :disappointed:

#13

…for Pong?

#14

Why not? Balls, paddles and anything else can hypothetically have some sort of animation, mm? :smiley:
image

Or in some cases balls can become fireballs. :laughing:

#15

but sprites with depth buffer enabled work just like a quad mesh. Just with an atlas instead of a simple texture. The key here is discard. Discarding is the solution right now, with its minuses, but it works, especially in the case of pixelart where there are no smooth transitions into transparency. But of course it would be nice to have control over how the objects are sorted.

2 Likes
#16

As well as I said that I’ll record a video asap I’m back to the machine, I’ll try the solution with discard; and let you guys know. Although the question is still open, because opaque sprites remain as a problem.

In addition, discard; solution although sounds reasonable, it hinders the fact that the depth-sorting order is not available to be modified, meanwhile Defold is very well though and super efficient engine :slightly_smiling_face:

#17

I think it’s something we want to add pretty soon, so I might just go ahead and add a design for it next week

5 Likes
#18

Here are the videos:
The problem I originally faced with the blending and depth_test and depth_sort

After discard; was applied in the fragment shader for both sprites and labels