Creating a native component

Questions
#1

Hi All,

100% Defold newbie here. I heard about it some years now, and now I’m considering it for a project.

My use case is to embed an emulator in my app and use Defold to create the UI and control the emulation i.e.:

  1. Create an instance of the emulator (it must support more than one running at the same time)

  2. Load a game ROM

  3. Start, stop and pause the emulation

  4. Save and load snapshots

The emulator is available as a dynamic link library, and I already can load it, resolve the symbols I need, and call into its API. By looking at the documentation, I assume it should be available in the editor as a component that I can attach to game objects.

The component will have to update a texture that will then be rendered by Defold via the game object it’s attached to. It will also have to push audio to the hardware audio device, preferrably via a pull API.

So my questions are:

  1. Is that assumption correct?

  2. If no, how should I approach the problem? I’m ok at changing Defold’s source code to add a new component type

  3. If yes, is it possible to create a new component type in a native extension?

Links to example code are more than welcome. I’ve already taken a look at some extensions but none seem to do what I need.

Thanks in advance for any help,

Andre

1 Like
#2

Just to clarify: I can do it from C++ code, what I need is to wrap that C++ code as a Defold component.

#3

Not exactly. Components in Defold are things such as Sprite, Tilemap, Model, Mesh etc. You can create your own components in extensions (Spine and Rive are two examples) but I do not see the need for it.

The closes example I can think of is actually a video player extension. Here’s a basic MPEG1 extension:

The extension creates a new Lua module with an API to interact with the underlying C code for the mpeg decoder. You use this API to decode a video frame, get the bytes and set those on a mesh. This is very similar to what you would do with an emulator.

2 Likes
#4

Thank you for your input.

Not exactly. Components in Defold are things such as Sprite, Tilemap, Model, Mesh etc. You can create your own components in extensions (Spine and Rive are two examples) but I do not see the need for it.

I understand. I thought about components because the way they’re described in the Building Blocks page:

Components are entities that are put in game object to give them visual, audible and/or logic representation in the game. They are typically used to create character sprites, script files, add sound effects or add particle effects.

The closes example I can think of is actually a video player extension. Here’s a basic MPEG1 extension:

Ah I somehow missed that one. Thanks a lot, it’s indeed very similar to what I need. I have a couple of doubts though:

  1. I understand that I can use the HBuffer in Lua to update a dynamic texture, is that right?

  2. I also assume that I can attach the above texture to a sprite, so I can transform and render it.

  3. The OnAudioSamplesDecoded callback is not implemented, and I couldn’t find an extension that does it.

    • FMOD is complex, maybe I should explore it more? I believe it’s just a native player though.

    • Debeat doesn’t seem to handle native audio sources.

    • Blip and DecodeOgg just save their results to WAV.

    • OpenAL and MOD Player use a native library to play audio.

I see that I can use an audio library, and I used miniaudio myself in the past. However, I’d rather change Defold to allow extensions to interact with its audio stream in order to support all the platforms Defold itself supports.

I wonder if I could create a SoundInstance for raw PCM audio frame streaming, and how I could expose it to extensions? All I need is a way to register a callback that is called whenever the engine needs more audio, so I can return the audio frames from the emulator.

Thanks again for taking time to respond, and thanks in advance for any additional help.

#5

Yes, this is exactly what the example included with the video player does!

In the most recent version of Defold we added support for sound streaming. I have not had time to update the video player extension to use it. I think this would solve your problem too.

1 Like
#6

Yes, this is exactly what the example included with the video player does!

Ah yes, I took a look at the included example and could see this:


-- get a video frame and update the texture

local videoframe = mpeg.get_frame(self.video)

if videoframe then

    local videoheader = { width=info.width, height=info.height, type=resource.TEXTURE_TYPE_2D, format=resource.TEXTURE_FORMAT_RGB, num_mip_maps=1 }

    local path = go.get("#sprite", "texture0")

    resource.set_texture(path, videoheader, videoframe)

end

The emulator can change the video frame resolution, e.g. when the game changes the video mode. I can do either of:

  1. Create a texture with the maximum size the emulator could use, and then specify the UV coordinates when blitting the texture.

  2. Recreate the texture with the new dimensions every time it changes the size (not really an issue, this doesn’t happen often)

Is any of those possible?

In the most recent version of Defold we added support for sound streaming. I have not had time to update the video player extension to use it. I think this would solve your problem too.

That’s great! I took a look at the source code and could find a callback system in place. However, the only callbacks supported are related to rendering:


typedef enum ExtensionCallbackType

{

    EXTENSION_CALLBACK_PRE_RENDER,

    EXTENSION_CALLBACK_POST_RENDER,

} ExtensionCallbackType;

I’m very likely looking at the wrong place, but I couldn’t find an audio streaming API anywhere.

Thanks again!

Andre

#7

Sure, you can create a texture at runtime:

Yep, wrong place! :slight_smile: I talked with @JCash and we haven’t created any public APIs yet. We would like to have a callback based solution where an extension can register a callback when a sound needs additional decoded sound data. I don’t know when this system will be available though.

#8

Sure, you can create a texture at runtime:

Sorry, I should’ve been more specific in my question. I’ve found go.set in the meanwhile though, which I believe I can use to set the sprite’s texture to the new one?

I don’t know when this system will be available though.

Oh ok. Does Defold accept PRs? I could give it a try with a bit of guidance: where to hook it, and how to expose it to the native extensions. I’m a experienced developer in both C/C++ and Lua.

Thank you again for the time spent answering my doubts.

#9

Yes we do. But let’s coordinate with @JCash first! He’ll answer here soon.

#10

Yes, this is likely a good candidate for contributions.
Step one is to create a ticket, and also write a design document around the topic.
We want to make sure that we don’t miss any relevant features that we want from this.
I’ve started the ticket and design here, with my current thoughts and requirements.

1 Like
#11

We also have documentation for contributing

and how to build the engine

and how to run the extender locally (our cloud build server).

1 Like