Haxe + Defold = ❤️

nadako · February 23, 2019, 5:10pm

Hi, Defold Community!

I’d like to present my hxdefold library that allows for using Haxe programming language with Defold. It’s actually not a new thing, but I never really announced it and I just updated it to the latest Defold API and polished it a bit for easier start.

For those who doesn’t know, Haxe is a neat and very powerful statically typed programming language that can compile to Lua, among dozen of other targets. So I thought - why not try to make it work with Defold?

What originally started as a quick experiment now looks more and more like an usable library, so here I am. Feel free to give it a try and don’t hesitate to ask questions and/or report issues (preferably through GitHub), but please abstain from programming language flame wars!

Oleg_The_Evangelist · February 25, 2019, 10:57am

we’re quite sensible here to not go into flame wars. But I guess it makes sense to talk about some cool use cases about Haxe in Defold context?

I know some teams used Haxe to easily run the same code in Defold client and the server. And I know there’s a way to translate Action Script code to Haxe and then run it in Defold via hxdefold.

Any other interesting use cases for it?

nadako · February 25, 2019, 11:47am

That’s a reasonable question. To be perfectly honest, I’m not really a Defold user, I just thought it was fun to try to fuse Haxe/Lua and Defold since they came out nearly at the same time, to get at least some “battle testing” for the new Haxe target.

As for the use cases, as you said, code sharing is a good use-case for Haxe in general, since you can compile your platform-agnostic business logic into different targets to reuse it on client/server or between different clients (the last one might not be very relevant here, since Defold/Lua can already be compiled to many platforms, but still this opens some interesting possibilities for testing and tooling).

Besides that, I would probably use Haxe rather than Lua for a larger code base even within single compilation target, just because it’s statically typed with a really expressive type system as well as the super powerful macro system for all kinds of validations and code generation (DRY to the extreme, e.g. validate static data directly from type definitions, without writing extra schemas).

It might be worth saying that Haxe/Lua output might look quite dirty if one is expecting 1:1 “transpilation”: it compiles into a single Lua module with quite some support code, because the basic types like String or Array in Haxe is very different from Lua counterparts. But in the end I find the compiled version of the actual code that you write in Haxe quite clean and optimized.

there’s a way to translate Action Script code to Haxe and then run it in Defold via hxdefold.

That is definitely possible, but might be quite involved as the as3hx tool is not perfect. OTOH we use a (heavily) modified version of it at work that actually produces a compileable and working haxe code from AS3 (which is still the primary source for now).

Oleg_The_Evangelist · February 25, 2019, 11:53am

Nice! Any plans to open source your fork of as3hx btw?

And thanks for your contribution! I’ve found your Patreon. Let me mention it here, in case Defold community decides to motivate you do something extra for us https://www.patreon.com/nadako

nadako · February 25, 2019, 11:57am

Well I don’t know It’s so dirty that I’d be ashamed to bring it into public, and it probably only works for our code base anyway. We’ll be developing a different converter very soon for actual migration to Haxe as a primary source and that thing will most probably be open.

AGulev · February 26, 2019, 8:22pm

insraq · March 3, 2019, 6:11pm

Having worked with Unity and Defold. One thing that I wish Defold had was support for static typing languages. Lua’s dynamic nature makes it less enjoyable to work with compared to C#. And I would argue a static type language is a better choice for a scripting in game engine because:

API navigation. Game engine usually has quite a big API surface. Navigating through API in a static language is such a breeze compared to a dynamic language.
Type check. For game development, the feedback loop is naturally not as fast . Therefore, it would be nice to discover more problems in compile time (or better, IDE would usually identify the problem while you are writing it). Defold’s live reload sort of makes the feedback loop faster, but still, with static typing, it would be better.
Code Confidence. Unit tests are important if you don’t want to accidentally break something in a dynamic language. However, let’s face it, we rarely write unit tests for games. Static type languages naturally gives more confidence.

That’s why I am super excited about hxdefold! After using it for a few days, I am super impressed. I have had a great experience when moving from JavaScript to TypeScript. And my experience with hxdefold largely follows that except for:

Haxe <-> Lua: because TypeScript is designed to be a JavaScript superset, the transition is super easy, even with large existing code base. You can gradually add types. Haxe is not designed to be a “typed Lua”. Therefore, it’s tricky to have a mixed of Haxe and Lua in your code base. This is strictly not a hxdefold problem, but rather a design choice of Haxe
Message: Defold’s message passing mechanism is quite dynamic and works with a dynamic language in mind. Trying to make it type safe it tricky. I like the way hxdefold implements Message<T>. However, it is quite verbose compared to C#. In Unity, you usually just define a function and invoke it. Of course, some times you have to use an event system (like UnityEvent) and in that case, it’s pretty much similar to hxdefold. However, since you cannot invode a function in another script, pretty much everything has to be done with message passing, which quickly get way too verbose.

nadako · March 3, 2019, 10:47pm

The goal here is to provide a no-overhead, but fully type-safe 1:1 mapping to the native Defold API, so I added Message<T> that you pass instead of strings/hashes in the pure Lua. It is actually a hash at run-time, so, zero overhead

But I agree that it is a bit verbose even in pure Lua (IMO it’s even worse, because they do if-then chains instead of switch). The good news is that with Haxe we can easily build some kind of high-level framework on top of this that will generate all the boilerplate, for example:

class MyScript {
 @:msg function heal(amount) {
   // handle "heal" message
 }
 @:msg function damage(amount) {
   // handle "damage" message
 }
}

and with a simple macro we can automatically generate (at compile-time) the on_message function like:

// macro-generated
function on_message(message, data) {
  if (message == "heal") this.heal(data);
  else if (message == "damage") this.damage(data);
}

and for sending these message we could have some (also macro-powered) api like:

someObject.msg(MyScript).damage(10)

that would compile into simple msg.post(someObject, "damage", 10). Of course everything will be type-checked at compile-time, with support for IDE services.

Stuff like this is very easy in Haxe, but to actually design a good high-level “framework” like that one needs to have some real experience with Defold and get a feeling of how things can be done better with a statically typed language. Unfortunately, so far I don’t have this kind of experience so I don’t want to introduce first thing that came into my mind (like the above). But we can of course discuss it if people is interested and I can provide implementation. It’s really not hard at all!

insraq · March 4, 2019, 9:28am

I would say that your proposal looks very good! When I look at my game code, 99% of my on_message works like that. And those 1% can easily be refactored to follow this paradigm. I am not Haxe expert. But being able to generate the code via compiler macro and still have the static type check in editor sounds very impressive.

However, my concern would be:

Debug: the current way that hxdefold bridges to Defold engine (i.e. have one big main file and generate a bridging script) makes it quite hard to debug already. And without source map support, I am not sure whether the new @:msg would make it even more difficult.
Scope: Defold script files have some quirks when it comes to scope (global/script/self/local). Compared to TypeScriptDefold project, which roughly transpile to match the Defold counterpart. hxdefold adds alevel of abstraction already, to match Haxe’s OOP model. Also Lua modules and hot reload needs some “global variable” workaround to function properly in Defold. All these add quite some complexity.

I see there are two design approaches:

Try to make transpile more transparent and match Defold Lua. This is in general how TypeScriptDefold works. I think it works reasonably well but I don’t think it’s feasible for hxdefold since Haxe and Lua are quite different at a fundamental level.
Try to design a high-level framework that hides all the details. It will essentially bend Defold towards a more traditional OOP design. hxdefold sort of already does this in some aspects. So I think it makes sense to have a different message dispatch mechanism. Of course, some low level quirks needs to be worked out (like hot reload) and debug will always be tricky without proper source map support.

nadako · March 4, 2019, 10:06am

Source mapping is something that I want to look into at some point. AFAIK, the mobdebug debugger used in Defold supports some kind of “line mapping” and we “just” need to provide a mapping function. It’s probably not as good as JS source maps, but it’s something. I’m not sure if Defold editor supports this though, but I think it should, as there are more compile-to-lua languages that people might want to use (e.g. MoonScript).

Regarding the @:msg there should be no problem actually, because it would be just a marker meta-data for a macro that collects messages from the class definition and generates the on_message function. The @:msg functions would be generated just like any normal function.

Tell me more about it! What quirks? I thought the hot-reload works by simply reloading all the code and the only state that’s guaranteed to be preserved is the self object, no?

Yeah I don’t see that happening. I acknowledge that 1:1 transpilation of syntax looks cute and clean in the examples, but I don’t think it matters and is worth pursuing in real world. What we really want is type-safety, boilerplate-less performance and code completion/navigation. That said, if something is technically can be done to make the generated code better/cleaner/easier-to-debug, we should of course work on that, please open suggestions on the Haxe issue tracker.

Well, hxdefold doesn’t really try to bend defold API to OOP in any way. Classes are used for scripts mostly because Haxe currently doesn’t support module-level functions. They are planned to be added in 4.1, but I don’t think I’ll be changing hxdefold to use that, because the current system has some advantages, like: Script<T> type parameter is used to strictly-type self for all the script callback functions, and you get the completion for the callback function signatures when you do override as a bonus

What I think can/should be done is a high-level framework on top of hxdefold that supports message method magic and other things.

vlaaad · March 4, 2019, 12:52pm

Can you please explain this? I don’t think I understood your argument here.

The way I see it, Defold’s Hot Reload provides a much faster feedback loop than any other engines with static typing because you don’t have to go through stop / build / run cycle, and instead can fix the code and continue with running game.

I think it would be very cool if Hot Reload could work with hxdefold (does it?) so we could get best of both worlds…

Mathias_Westerdahl · March 4, 2019, 1:07pm

One thing we’ve mentioned several times before, is to support Lua linters.
This will also help to some degree.

nadako · March 4, 2019, 1:12pm

Okay, can someone point me to a doc describing requirements and best practices for the hot reload so I can check if there’s a problem with it and hxdefold next time I get some free time?

britzl · March 4, 2019, 1:39pm

Hot-reload is described here: https://www.defold.com/manuals/hot-reload/

insraq · March 4, 2019, 2:08pm

Or specifically this section: Hot Reloading Modules

nadako · March 4, 2019, 2:27pm

Thanks, I’ll read that after work ^^

insraq · March 4, 2019, 2:31pm

Just an typical example of game dev: I have a Player object, I need to decrease the HP by 1.

In Unity, in your player component, you do:

public void DecreaseHealth(int health)
{
    ...
}

In the caller, you do:

DecreaseHaelth(1);

And oops, I spell “health” wrong, the IDE will give me a red underline and I correct the error.

In Defold, in your player script, you do:

function on_message(self, id, message, sender)
    if message_id == hash("decrease_health") then
        ...
    end
end

In the caller, you do:

msg.post("player#controller", "decrease_haelth", { by: 1 })

And oops, I spell “health” wrong. The game runs fine though, until I realize the health is not decreased. After some debugging (assuming I can reproduce the trigger easily) I finally find out that I spell “health” wrong.

So yes, Hot Reload makes the feedback loop faster. However, static type languages will tell you this error as fast as you type it (assuming you are using a decent IDE). And you are much less likely to make this error since there’s IntelliSense.

Linter can help. However, my problem with linter is that it does not discover all error and especially in this case, since message passing is “string-based”, it is impossible for linters to spot the problem.

AGulev · March 4, 2019, 2:53pm

Usually, I have a module msgs (or smth like this) and it looks like:

local M ={}
...
M.HEALTH_DECREASE = hash("msgs.HEALTH_DECREASE")
...
return M

I require this module everywhere where I wanna use messages. And Atom helps me with autocompletion. Even without autocompletion if I write wrong I will receive a message about a nil value for message_id.

nadako · March 4, 2019, 4:44pm

FWIW I made a helper for that in hxdefold: http://hxdefold.github.io/hxdefold/defold/support/MessageBuilder.html

I like the @:msg function idea still and probably will implement a quick prototype for that soon-ish

insraq · March 4, 2019, 4:55pm

I have similar workarounds but I don’t store hashed value. Storing hashed value should improve performance but how do you use it when sending messages?
You cannot do the following right?

msg.post("player#controller", M.HEALTH_DECREASE, { by: -1 })