Having worked with Unity and Defold. One thing that I wish Defold had was support for static typing languages. Lua’s dynamic nature makes it less enjoyable to work with compared to C#. And I would argue a static type language is a better choice for a scripting in game engine because:
- API navigation. Game engine usually has quite a big API surface. Navigating through API in a static language is such a breeze compared to a dynamic language.
- Type check. For game development, the feedback loop is naturally not as fast . Therefore, it would be nice to discover more problems in compile time (or better, IDE would usually identify the problem while you are writing it). Defold’s live reload sort of makes the feedback loop faster, but still, with static typing, it would be better.
- Code Confidence. Unit tests are important if you don’t want to accidentally break something in a dynamic language. However, let’s face it, we rarely write unit tests for games. Static type languages naturally gives more confidence.
That’s why I am super excited about hxdefold! After using it for a few days, I am super impressed. I have had a great experience when moving from JavaScript to TypeScript. And my experience with hxdefold largely follows that except for:
-
Haxe <-> Lua: because TypeScript is designed to be a JavaScript superset, the transition is super easy, even with large existing code base. You can gradually add types. Haxe is not designed to be a “typed Lua”. Therefore, it’s tricky to have a mixed of Haxe and Lua in your code base. This is strictly not a hxdefold problem, but rather a design choice of Haxe
-
Message: Defold’s message passing mechanism is quite dynamic and works with a dynamic language in mind. Trying to make it type safe it tricky. I like the way hxdefold implements
Message<T>. However, it is quite verbose compared to C#. In Unity, you usually just define a function and invoke it. Of course, some times you have to use an event system (like UnityEvent) and in that case, it’s pretty much similar to hxdefold. However, since you cannot invode a function in another script, pretty much everything has to be done with message passing, which quickly get way too verbose.
The goal here is to provide a no-overhead, but fully type-safe 1:1 mapping to the native Defold API, so I added Message<T> that you pass instead of strings/hashes in the pure Lua. It is actually a hash at run-time, so, zero overhead 
But I agree that it is a bit verbose even in pure Lua (IMO it’s even worse, because they do if-then chains instead of switch). The good news is that with Haxe we can easily build some kind of high-level framework on top of this that will generate all the boilerplate, for example:
class MyScript {
@:msg function heal(amount) {
// handle "heal" message
}
@:msg function damage(amount) {
// handle "damage" message
}
}
and with a simple macro we can automatically generate (at compile-time) the on_message function like:
// macro-generated
function on_message(message, data) {
if (message == "heal") this.heal(data);
else if (message == "damage") this.damage(data);
}
and for sending these message we could have some (also macro-powered) api like:
someObject.msg(MyScript).damage(10)
that would compile into simple msg.post(someObject, "damage", 10). Of course everything will be type-checked at compile-time, with support for IDE services.
Stuff like this is very easy in Haxe, but to actually design a good high-level “framework” like that one needs to have some real experience with Defold and get a feeling of how things can be done better with a statically typed language. Unfortunately, so far I don’t have this kind of experience so I don’t want to introduce first thing that came into my mind (like the above). But we can of course discuss it if people is interested and I can provide implementation. It’s really not hard at all!
I would say that your proposal looks very good! When I look at my game code, 99% of my on_message works like that. And those 1% can easily be refactored to follow this paradigm. I am not Haxe expert. But being able to generate the code via compiler macro and still have the static type check in editor sounds very impressive.
However, my concern would be:
-
Debug: the current way that hxdefold bridges to Defold engine (i.e. have one big main file and generate a bridging script) makes it quite hard to debug already. And without source map support, I am not sure whether the new
@:msgwould make it even more difficult. -
Scope: Defold script files have some quirks when it comes to scope (global/script/self/local). Compared to TypeScriptDefold project, which roughly transpile to match the Defold counterpart. hxdefold adds alevel of abstraction already, to match Haxe’s OOP model. Also Lua modules and hot reload needs some “global variable” workaround to function properly in Defold. All these add quite some complexity.
I see there are two design approaches:
-
Try to make transpile more transparent and match Defold Lua. This is in general how TypeScriptDefold works. I think it works reasonably well but I don’t think it’s feasible for hxdefold since Haxe and Lua are quite different at a fundamental level.
-
Try to design a high-level framework that hides all the details. It will essentially bend Defold towards a more traditional OOP design. hxdefold sort of already does this in some aspects. So I think it makes sense to have a different message dispatch mechanism. Of course, some low level quirks needs to be worked out (like hot reload) and debug will always be tricky without proper source map support.
Source mapping is something that I want to look into at some point. AFAIK, the mobdebug debugger used in Defold supports some kind of “line mapping” and we “just” need to provide a mapping function. It’s probably not as good as JS source maps, but it’s something. I’m not sure if Defold editor supports this though, but I think it should, as there are more compile-to-lua languages that people might want to use (e.g. MoonScript).
Regarding the @:msg there should be no problem actually, because it would be just a marker meta-data for a macro that collects messages from the class definition and generates the on_message function. The @:msg functions would be generated just like any normal function.
Tell me more about it! What quirks? I thought the hot-reload works by simply reloading all the code and the only state that’s guaranteed to be preserved is the self object, no?
Yeah I don’t see that happening. I acknowledge that 1:1 transpilation of syntax looks cute and clean in the examples, but I don’t think it matters and is worth pursuing in real world. What we really want is type-safety, boilerplate-less performance and code completion/navigation. That said, if something is technically can be done to make the generated code better/cleaner/easier-to-debug, we should of course work on that, please open suggestions on the Haxe issue tracker.
Well, hxdefold doesn’t really try to bend defold API to OOP in any way. Classes are used for scripts mostly because Haxe currently doesn’t support module-level functions. They are planned to be added in 4.1, but I don’t think I’ll be changing hxdefold to use that, because the current system has some advantages, like: Script<T> type parameter is used to strictly-type self for all the script callback functions, and you get the completion for the callback function signatures when you do override as a bonus
What I think can/should be done is a high-level framework on top of hxdefold that supports message method magic and other things.
Can you please explain this? I don’t think I understood your argument here.
The way I see it, Defold’s Hot Reload provides a much faster feedback loop than any other engines with static typing because you don’t have to go through stop / build / run cycle, and instead can fix the code and continue with running game.
I think it would be very cool if Hot Reload could work with hxdefold (does it?) so we could get best of both worlds…
One thing we’ve mentioned several times before, is to support Lua linters.
This will also help to some degree.
Okay, can someone point me to a doc describing requirements and best practices for the hot reload so I can check if there’s a problem with it and hxdefold next time I get some free time?
Or specifically this section: Hot Reloading Modules
Thanks, I’ll read that after work ^^
Just an typical example of game dev: I have a Player object, I need to decrease the HP by 1.
In Unity, in your player component, you do:
public void DecreaseHealth(int health)
{
...
}
In the caller, you do:
DecreaseHaelth(1);
And oops, I spell “health” wrong, the IDE will give me a red underline and I correct the error.
In Defold, in your player script, you do:
function on_message(self, id, message, sender)
if message_id == hash("decrease_health") then
...
end
end
In the caller, you do:
msg.post("player#controller", "decrease_haelth", { by: 1 })
And oops, I spell “health” wrong. The game runs fine though, until I realize the health is not decreased. After some debugging (assuming I can reproduce the trigger easily) I finally find out that I spell “health” wrong.
So yes, Hot Reload makes the feedback loop faster. However, static type languages will tell you this error as fast as you type it (assuming you are using a decent IDE). And you are much less likely to make this error since there’s IntelliSense.
Linter can help. However, my problem with linter is that it does not discover all error and especially in this case, since message passing is “string-based”, it is impossible for linters to spot the problem.
Usually, I have a module msgs (or smth like this) and it looks like:
local M ={}
...
M.HEALTH_DECREASE = hash("msgs.HEALTH_DECREASE")
...
return M
I require this module everywhere where I wanna use messages. And Atom helps me with autocompletion. Even without autocompletion if I write wrong I will receive a message about a nil value for message_id.
FWIW I made a helper for that in hxdefold: http://hxdefold.github.io/hxdefold/defold/support/MessageBuilder.html
I like the @:msg function idea still and probably will implement a quick prototype for that soon-ish
I have similar workarounds but I don’t store hashed value. Storing hashed value should improve performance but how do you use it when sending messages?
You cannot do the following right?
msg.post("player#controller", M.HEALTH_DECREASE, { by: -1 })
Yes, it should improve performance a bit, I am using it next way:
local msgs = require "modules.msgs"
...
msg.post("player#controller", msgs.HEALTH_DECREASE, { by: -1 })
...
Also it possible to register your go’s in some module with paths and do not use “player#controller”, something like:
local paths = require "modules.paths"
function init(self)
paths.register("player")
end
function final(self)
paths.remove("player")
end
and path.lua:
local M = {}
...
function M.register(name)
M[name] = msg.url()
end
function M.remove(name)
M[name] = nil
end
...
return M
And now we can use :
local msgs = require "modules.msgs"
local paths = require "modules.paths" --we can use one module here
...
if paths.player then --now we can check "if player exist"
msg.post(paths.player, msgs.HEALTH_DECREASE, { by: -1 })
end
...
Of course it is just a dirty example and of course, it’s possible to do better.
But what I wanna say, When I stared with Defold I hate Lua (I have exp. with AS3, C#, Haxe, Typescript before), but after some time I found it powerful and convenient for game development.
It is possible to avoid most of your concerns just using code architecture decisions. But it needs time to found them.
But… I like how Haxe help to avoid the same problems out of the box - it’s a really powerful language and Dan did a great job. If you like Haxe and don’t wanna spend your time to learn Lua, Haxe is a great solution for you, I think.
Also, it’s possible to use TypeScript, maybe it would be interesting for you.
You could do something similar in Defold using a Lua module:
-- message_poster.lua
local M = {}
M.HEALTH_DECREASE = hash("msgs.HEALTH_DECREASE")
function M.decrease_health(amount)
assert(amount)
msg.post("player#controller", M.HEALTH_DECREASE, { by: amount })
end
return M
By putting all of the message posting in a module you’d at least get code completion for the different functions for sending messages. You wrap this into an even nicer module that takes care of both posting messages and register handlers for them.
BTW, in case you are not aware, hot reload does not work with hxdefold. It has something to do with the lua modules.
Yes. The Defold TypeScript wrapper is a thin layer. And the widely use of any type sort of makes it less exciting. I love TypeScript and I never look back since converting to TypeScript years ago.
And I agree there are lots of ways you can structure your Lua scripts to make it better, but still it never feels as good as a static type language. Maintaining Lua is much harder, especially you have lots of people touching the code. My Dota Mod was quite hard to maintain by myself already and quickly became impossible to maintain after I several contributors joined.
Also, as you mentioned, another nice point of static type language is that it works “out of the box”. And you always feel “safe” because of Type/IntelliSense. Also you can refactor with confidence.
Well the point is not having to write the these functions by hand
Anyway, it is nice to have options for every taste and I totally should actually try creating something with Defold.