Flix - Procedural training and Movie making

But what remains for this to be completely dynamic so that you don’t have to create all of these up-front? I’m trying to understand what we could do to simplify life for you.

As per a number of other threads on the forums, there is no real way to create a new GameObject, that can then have a component (new also) added to it. My understanding (from reading the various threads) is that factories create instances, and thus data for them is shared (some is not, like position etc but components like meshes are shared).

I did try using factories, but as expected on instance 2… modifying the mesh on the instanced go crashed the running app. I get why this is, and I dont know how deep a problem that would be to add. My meshpool was a way around that, but with a tool that will potentially have hundreds of characters, and objects in the scene, I dont think that will scale well at all :slight_smile:

And, again, this is probably not a common use case. I dont expect a game engine to necessarily support this kind of resource usage, thus I dont really want to request it :slight_smile:
I’ll find a way around it - there are many different things I want to try over the next few days. For example:

  • generate a collapsed mesh (at runtime) by composing in the transforms for more static objects.
  • use something like ozz animation lib to handle anim, thus the bones are not needing to be managed in the go structure itself
  • try a simple backend renderer that has some of these (like maybe sokol or something) that I can layer in as a render pass into Defold.
  • use another engine entirely for the render pass embedded in the app :wink: which would be nuts, but Ive done it with other systems before (like C# WPF running Unigine within it ).

There some interesting things to look at. Will draw up some ideas, throw away any that just are non-sensical, and knock out some tests for the remaining ones to validate the design. Then move on :slight_smile:

Yes you can’t create go with components in runtime. You should use factories.

But you can create buffer resource for mesh in runtime.

local function water_create_default_native_buffer()
	return buffer.create(1, {
		{ name = hash("position"), type = buffer.VALUE_TYPE_FLOAT32, count = 3 },
		{ name = hash("normal"), type = buffer.VALUE_TYPE_FLOAT32, count = 3 },
		{ name = hash("texcoord0"), type = buffer.VALUE_TYPE_FLOAT32, count = 2 },
	})
end

---@return BufferResourceData
function M.water_create_new_buffer()
	M.water.idx = M.water.idx + 1
	local name = "/runtime_buffer_water_" .. M.water.idx .. ".bufferc"
	local new_buffer = resource.create_buffer(name, { buffer = water_create_default_native_buffer() })

	---@class BufferResourceData
	local buffer_resource = {}
	buffer_resource.name = name
	buffer_resource.buffer = new_buffer
	buffer_resource.free = false

	return buffer_resource
end
go.set(e.water_go.mesh, "vertices", e.water_buffer.buffer)
2 Likes

Im not sure you have understood the problem. I have a geomextension lib (shared a couple yrs ago) that does this. But you cannot apply new meshes to instances from a factory. I’ve tried this many times over the years because components cant be added to the instance at runtime.

The buffers would be more useful (for this instance) if I could create a mesh component and attach it to a go instance. That would be a nice way to do this, but I also suspect its a much more complicated problem since resources are all precompiled at build time.

Sorry that was a little unclear. Specifically, if I instance a go with an attached mesh, that mesh buffer resource handle is shared with all instances. Thus, you cannot replace the mesh with a buffer. If you attempt to do so, on the second instance you try it on, it can crash (because of the reference held from the first one). This was mentioned above.

For some references to the above. Here are some of the forum posts that relate to this. Some quite old, but this is kinda a known limitation for runtime resource usage:

And there are other related ones.

Also just for some more clarity. My geom extenstion (this one a little modified) does the following when assigning new buffers to a mesh.

------------------------------------------------------------------------------------------------------------

function geom:makeMesh( goname, indices, verts, uvs, normals )

	if(#indices <= 0) then 
		print("[Error] Invalid indices count.")
		return 
	end
	
	local res = go.get(goname, "vertices")
	local iverts = #indices

	local meshdata = {}
	-- positions are required (should assert or something)
	tinsert(meshdata, { name = hash("position"), type=buffer.VALUE_TYPE_FLOAT32, count = 3 } )
	if(normals) then tinsert(meshdata, { name = hash("normal"), type=buffer.VALUE_TYPE_FLOAT32, count = 3 } ) end
	if(uvs) then tinsert(meshdata, { name = hash("texcoord0"), type=buffer.VALUE_TYPE_FLOAT32, count = 2 } ) end
	
	local meshbuf = buffer.create(iverts, meshdata)

	geomextension.setbufferbytesfromtable( meshbuf, "position", indices, verts )
	if(normals) then geomextension.setbufferbytesfromtable( meshbuf, "normal", indices, normals ) end
	if(uvs) then geomextension.setbufferbytesfromtable( meshbuf, "texcoord0", indices, uvs ) end 
		
	-- set the buffer with the vertices on the mesh
	resource.set_buffer(res, meshbuf)
end

------------------------------------------------------------------------------------------------------------

So the problem isnt being able to generate mesh buffer data, but trying to replace the mesh component handle so that it has a new handle. This might be doable in C++ but I’m not certain of that.
If there are examples/methods that can do this, I would be very welcome to suggestions.

I think this specific comment shows the main problem:

local resource_name = "/my_cloned_buffer.bufferc" -- make this unique for each resource!
This means there needs to be a physical resource file for each res that needs to be attached/modified for each instance.

Note: The code includes some helper funcs that copy tables into buffer data for consumption by the buffer.

1)You have factory.
Factory create go with mesh.
2) From url to this mesh. You can get vertices.

local res = go.get("#mesh", "vertices")

If you print vertices you will get something like this
“vertices: “/world/game/go/chunk/chunk.buffer”\n”

3)You can change this buffer. Now all meshes that have vertices “vertices: “/world/game/go/chunk/chunk.buffer”\n” is changed.

resource.set_buffer(res, buf)

4)Also you can change vertices field for you mesh instance.

    -- assign cloned buffer to a mesh component
go.set("/go#mesh", "vertices", buf)```

So for example. You have one factory. You create 10 go. For every go you change vertices. 
Now you have 10 unique meshes with different meshes.

No, this is only name. It was physical resource if this buffer is created by bob when build game.

But you can create this resources in runtime. You will only need string name with .bufferc ending.
This will not create a new file. This .bufferc will be in memory

Ok. Im not sure I understand. How do I make new resource buffers at runtime
Those other bufferc files do not exist.

If you look at my code above. This is exactly what you suggest, but you cannot change the mesh because the resource (go.get("#mesh", “vertices”) ) is shared on all instanced go’s.

Hrm. This is only a url handle? So in the code above I should be able to set res = “randomname.bufferc” ?

< Update > Just tried this (in above code)

local res = "testbuffer_"..string.format("%d", ctr)..".bufferc" -- go.get(goname, "vertices")

Error:

Not really sure how this could work.

1 Like

Yes, look at resource.create_buffer()

NEW: (#7214 ) Create buffer in runtime
Added new function resource.create_buffer to create buffer resources dynamically in runtime. The function takes a buffer object created by other buffer resources, from the C api or via the lua buffer.create function.

1 Like

Again, I think you are not really understanding? I can create buffers no problem. Look at the above code please. It does exactly what you suggest.

However, the problem is attaching those buffers to an instanced go. If you use:
local res = go.get("#mesh", "vertices")
this fetches a resource handle that is a compiled bufferc. Which is done at build time.

If (and I think this is what you are saying) you can make a new resource at runtime with a valid resource handle then that is what should be used. However, just making up a name wont work, because the registry wont have that file? Maybe in C++ I can add something that registers a new file resource that then can be used? Again, it is not clear how your example could possibly work.

Note: In the api there isnt really a “create resource” call that generates a new runtime resource for use. The closest thing I can see is in the C++ api where there is an AddFile, but Im not sure that is whats needed.

More Notes: I was thinking probably the easiest thing that would potentially provide a simple solve is a copy method in the C++ api, that allows the copying of a resource in mem. And then this would be fairly simple to apply (maybe?)

1 Like

You don’t need new file.

You create runtime resource. This is in memory.
This resource will be alived while you have references for it.(If I understand right)

In this example buffer is clonned. But you can use new buffer.

When you create a new resource, you set content for it. So you don’t need file.

local res = resource.get_buffer("/my_buffer_path.bufferc")
    -- create a cloned buffer resource from another resource buffer
    local buf = reource.create_buffer("/my_cloned_buffer.bufferc", { buffer = res })

This is video of my voxels game. I use one factory for chunk. And I can have unique meshes for every chunk. I am creating new buffer resources in runtime.

3 Likes

Try it, when you write some code you will see how it worked:)

1 Like

Hrm. I think you are using different methods. This might be the problem. Im using buffer.create, which is different to resource.create_buffer.

Just to be clear, are you generating instanced go’s at runtime each with a separate mesh objects? Or are you created one mesh, with additional/other buffers on that mesh component? Because that is already something Im doing.

You absolute bloody legend!!! Thankyou. It was the commands I was using!!!
Ive changed it to this:

------------------------------------------------------------------------------------------------------------
local ctr = 0
function geom:makeMesh( goname, indices, verts, uvs, normals )

	if(#indices <= 0) then 
		print("[Error] Invalid indices count.")
		return 
	end
	
	--	local mesh = resource.load("/assets/gotemplate/temp.mesh")
	pprint(goname)
	local res = resource.get_buffer(go.get(goname, "vertices"))	
	-- create a cloned buffer resource from another resource buffer
	local newres = resource.create_buffer("/testbuffer_"..string.format("%d", ctr)..".bufferc", { buffer = res })	
	ctr = ctr + 1
	pprint(newres)
	local iverts = #indices

	local meshdata = {}
	-- positions are required (should assert or something)
	tinsert(meshdata, { name = hash("position"), type=buffer.VALUE_TYPE_FLOAT32, count = 3 } )
	if(normals) then tinsert(meshdata, { name = hash("normal"), type=buffer.VALUE_TYPE_FLOAT32, count = 3 } ) end
	if(uvs) then tinsert(meshdata, { name = hash("texcoord0"), type=buffer.VALUE_TYPE_FLOAT32, count = 2 } ) end
	--{ name = hash("color0"), type=buffer.VALUE_TYPE_FLOAT32, count = 4 }
	
	local meshbuf = buffer.create(iverts, meshdata)

	geomextension.setbufferbytesfromtable( meshbuf, "position", indices, verts )
	if(normals) then geomextension.setbufferbytesfromtable( meshbuf, "normal", indices, normals ) end
	if(uvs) then geomextension.setbufferbytesfromtable( meshbuf, "texcoord0", indices, uvs ) end 
		
	-- set the buffer with the vertices on the mesh
	resource.set_buffer(newres, meshbuf)
	go.set(goname, "vertices", newres)
end

------------------------------------------------------------------------------------------------------------

And it is mostly working - need to obviously change the ctr naming. Will do some testing, I will need to pump up the buffer and mesh limits now too. Thankyou for helping, I was really struggling with how you could possibly have done it without a resource!

Thanks again. And I really appreciate the patience and help.
Also, I apologize for not realizing the code I was using was different. Very sorry. Thankyou again.

5 Likes

@d954mas - Again. Cant thank you enough. It looks like this should work.
Im now having material instancing issues, but thats because many of my naming conventions have moved around so they are referencing the wrong objects. A little piccy:


Yes thats an MQ-9 in the bg. :slight_smile:

2 Likes

Welcome, I Iike to read your posts, you make cool things​:+1::+1:

3 Likes

Thanks mate. You make some astonishing projects yourself. You are quite an inspirational developer, so thanks if it hasnt already been said too much :slight_smile:

2 Likes

It’s thanks to @d954mas that we’ve improved the dynamic buffer system quite a bit during the last year :slight_smile:

6 Likes