How does Defold handle multithreading?

#1

I couldn’t find anything on how Defold handles C++ threads in the documentation. If someone could point me to the right location I’d appreciate it.

I’m familiar with the web/browser model where you can pass some work from otherwise single threaded javascript to a javascript web worker which is doing its own thing off the main thread and can then pass things back. The main concept here being you write things like UI on main-thread and pass the rest on web worker if that is available.

But I’m quite new to the concept of Single threaded language passing instructions to a different multi-threaded language on anything but a “theoretical level” :slightly_smiling_face: .

TLDR;
I’m looking for:

  • which tasks are relayed from Lua to be handled on the C++ side
  • does C++ side already use something like “thread detection” and spreads things that can be on different thread to available threads
  • simplified overview but for Defold tech architecture
  • If it is already using something like my “dream code for multi-threading” where you write asynchronous code with something like Lua coroutines and it then messages the C++ which spreads concurrent tasks on available threads
1 Like

#2

If you write a native extension to do something you can have threads to do work. If you do need to do things which would block normal gameplay you should try to do it in NE threads, on some platforms like HTML5 this won’t help though because everything is forced single thread due to security requirements.

Otherwise Lua coroutines are not threads, there is nothing special done with that as far as I know. Would be cool if it was possible! Like ways to write async Lua containers that are run on their own actual threads so as to not block the main thread and can pass data as needed.

Some parts may be threaded but currently others like built in audio are not. You should generally assume things done in Lua are not threaded.

I think http.request() calls are threaded. Some resource loaded may be threaded but even with async set it can still lag your game, even audio will pause/glitch in some cases since it’s not threaded yet.

Of course the source is available now so you can read it yourself to see how it actually all works.

Something like the “simplified overview” to compliment https://defold.com/manuals/application-lifecycle/ would be cool to see.

4 Likes

#3

I actually was going to link to the documentation for the dmThread, only to find I still hadn’t added it to the SDK.
Added https://github.com/defold/defold/issues/4835 for this.

I like the idea of a more schematic overview of the engine, to help users/developers understand what’s going on under the hood.

Currently, there are only a few things that are threaded:
http requests, live update, some graphics requests (e.g. texture uploads),

Having the “UI on the main thread” mindset here is still valid, in our case it’s the Lua context that lives on the main thread, all callbacks need to be executed on the main thread.

We aim to do more threading though, it just hasn’t been too high on our road map.
I think first up on the list is to see if we can offload sound to a new thread, or perhaps resource loading.

7 Likes