Billboard shader, Model matrix?

Looking forward to it!

Is there something about sprites vs models which would make this not work with sprites?

:thinking:

Ok, using model as billboard is not that hard

 vec4 p = mtx_worldview * vec4(0.0, 0.0, 0.0, 1.0 /*scale*/) + vec4(position.xyz, .0);

did the trick.

But I stuck with sprites. As sprite shader has view_proj * vec4(position.xyz, 1.0) and I have no idea how defold calculates go rotations here.

In theory we can pass rotation (or reversed rotation) for each billboard object to shader (via sprite.set_constant) but what way we should apply this rotation? Huh.

That would break batching (I think?) which is a no-go, this has to be able to be done entirely in shaders for performance reasons and keep performance. I think point sprites are what are supposed to be used for billboard particles and maybe even stuff like foliage / grass.

I still don’t fully understand your case. Maybe you should make a video with wrong behaviour of current (build-in) particle shader or issue with rendering tree (made of sprites?).

Main thing I want is efficient 3d particles for 3d games where player can move around a world. If you put a particlefx in 3d space the particles will always look in a fixed direction and cannot follow the player. I want things that are possible in other game engines.

1 Like

I agree. This was one of our most requested feature for Blastlands (that wasnt critical) as we used a lot of 3D rotations of objects but still wanted the particles to face camera.

4 Likes

Sorry to resurrect this thread, but I can’t seem to find the answer anywhere else on the forum. Would anyone happen to have a billboard material for models, based on the shader code above?

I’ve trying to turn this into a billboard material, but can’t get it to work.

Update: @Pkeod to the rescue! https://github.com/subsoap/defold-shader-examples

4 Likes

Let’s go back to that question.
The bottom line is this: is it REALLY necessary to pass labels, sprites and particles NOT as billboards?
All the games I’ve seen - they all use particles and labels as billboards.
It would make sense that this would be the default behaviour for sprites and particles in Defold!

How can this be resolved?
Pass in the vertex data the centre of the sprite.
For example, the vertex declaration now has the format position + texcoord.
Pass in each vertex additionally the centre of the sprite/particle (you know it anyway). This will align the sprite with the camera in the custom shader.
This solution is completely compatible with previous versions of the engine and does not break anything.

5 Likes

In order to speed up the solution to the problem from this thread, i have created a request for a feature: https://github.com/defold/defold/issues/6820
Please, support this request if you think i am right.

5 Likes

I’m now in a need for a billboard solution for sprite (and particles) in 3D world. I tried writing my own function that rotates the sprite towards camera position, but I am failing miserably all the time :frowning:

Do you guys have any working solution for sprites and how to use it?

Sprites should be easy now with custom vertex attributes + setting attribute data together with go.set. Particles is more difficult since we don’t have support for the setting, but it’s not impossible. You need to pass the center position of the primitive into the shader, perhaps a texture or something?

1 Like

What I wish to achieve is this effect (without changing sprite’s animation, I will be only slightly moving camera)

billboard rotation

The example from the @Pkeod Defold Shaders is as follows:

// Positions can be world or local space, since world and normal
// matrices are identity for world vertex space materials.
// If world vertex space is selected, you can remove the
// normal matrix multiplication for optimal performance.

attribute highp vec4 position;
attribute mediump vec2 texcoord0;
attribute mediump vec3 normal;

uniform mediump mat4 mtx_worldview;
uniform mediump mat4 mtx_view;
uniform mediump mat4 mtx_proj;
uniform mediump mat4 mtx_normal;
uniform mediump vec4 light;

varying highp vec4 var_position;
varying mediump vec3 var_normal;
varying mediump vec2 var_texcoord0;
varying mediump vec4 var_light;

void main()
{

    mat4 original_matrix = mtx_worldview;

    mat4 model_view;
    model_view[0] = vec4(length(original_matrix[0].xyz), 0.0, 0.0, 0.0);
    model_view[1] = vec4(0.0, length(original_matrix[1].xyz), 0.0, 0.0);
    model_view[2] = vec4(0.0, 0.0, length(original_matrix[2].xyz), 0.0);
    model_view[3] = original_matrix[3];

    vec4 p = model_view * position;

    var_light = mtx_view * vec4(light.xyz, 1.0);
    var_position = p;
    var_texcoord0 = texcoord0;
    var_normal = normalize((mtx_normal * vec4(normal, 0.0)).xyz);
    gl_Position = mtx_proj * p;
}

With modified to be simpler fp:


varying highp vec4 var_position;
varying mediump vec3 var_normal;
varying mediump vec2 var_texcoord0;
varying mediump vec4 var_light;

uniform lowp sampler2D tex0;
uniform lowp vec4 tint;

void main()
{
    // Pre-multiply alpha since all runtime textures already are
    vec4 tint_pm = vec4(tint.xyz * tint.w, tint.w);

    gl_FragColor = texture2D(tex0, var_texcoord0.xy) * tint_pm;
}

And it’s slightly ok, but it’s on a model and I don’t know how to make transparent pixels transparent and how to rotate it’s initial texture:
image

This I don’t understand:

Camera is not rotating in my game, but is slightly tilted and is only moving around (changing only position)

I think @Dragosha has an example somewhere, not sure.
I saw something in this issue True Billboards for 3D (label, sprite, particles components) · Issue #6820 · defold/defold · GitHub
vp hsader:

gl_Position = (view_proj * vec4(position.xyz - local_position.xyz, 1.0)) + (proj * vec4(local_position.xy, 0.0, 0.0));

materials:

attributes {
  name: "local_position"
  semantic_type: SEMANTIC_TYPE_POSITION
  element_count: 3
  normalize: false
  data_type: TYPE_FLOAT
  coordinate_space: COORDINATE_SPACE_LOCAL
  double_values {
    v: 0.0
    v: 0.0
    v: 0.0
  }
}
1 Like

Yes, this is the optimised version with full billboard (all axes)

Another variant:
here we remove the y-axis rotation towards the camera

highp vec4 base = vec4(position.x - local_position.x, position.y, position.z - local_position.z, 1.0);
highp vec4 local_pos = vec4(position.x - base.x, position.y - base.y, 0.0, 0.0);
gl_Position = (view_proj * base) + (proj * local_pos);

4 Likes

Thank you!

I started from model.material, added local_position as Vertex Attribute as defined by @AGulev .
I guess it will be a position of the billboard object, that I will need to set from script:

go.set(url, "local_position", go.get_position(url))

So for test, I set this material to the model, but I don’t see a texture in Editor, but it’s definitely behaving like billboard when rotated:

fp is simplified, maybe here is an erro?

varying highp vec4 var_position;
varying mediump vec3 var_normal;
varying mediump vec2 var_texcoord0;
varying mediump vec4 var_light;

uniform lowp sampler2D tex0;
uniform lowp vec4 tint;

void main()
{
    // Pre-multiply alpha since all runtime textures already are
    vec4 tint_pm = vec4(tint.xyz * tint.w, tint.w);

    gl_FragColor = texture2D(tex0, var_texcoord0.xy) * tint_pm;
}

EDIT. Oh! And I noticed, you use this material on sprites @Dragosha !

you don’t need to set local_position variable from script.

also in your shader missed var_texcoord0 = texcoord0; (throw UV coordinate from vertex shader to fragment shader)

Thank you!
It solves my issue :heart:

image

What is it then?

1 Like

Here is an atricle how it works:

SEMANTIC_TYPE_POSITION automatically setting up by engine.

But I’m not sure that it works with the model component. I tested only with sprites and particles. (model billboard? why?)

1 Like

Thank you!

No particular reason, I just tend to start every shader play with a model component, since I usually was using some data from textures, but here it’s just out of habit :smiley: