Will Defold support an official alternative language like Godot does?

Questions
41

I like Lua. I started using it in the Backend with OpenResty after discovering Defold.

1 Like
42

Yeah, I asked about code gen, since this makes F# and Clojure support impossible.

By implementing the API normally, it would have been compatible with all dotnet languages (for no extra work)

Since there is also Python and PHP supported, that would have been awesome.

Thanks for answering!

43

Not sure why that would be, as we generate wrappers that call our C/C++ api. If it works for C#, I don’t see why it wouldn’t work for more languages.

By implementing the API normally

Having the api written in C, and then let use our C++ and C# apis call that sounds pretty normal.
It certainly is compatible with C#.

Since there is also Python and PHP supported

What do you mean? Supported where?

44

I mean calling it via the same API. The whole point of .NET is to have a platform, that hosts different languages.

Nearly every single library and binding for C#, is also accessible for every other language on the platform.

As it is the case for JVM, both platforms use an intermediate language they compile to.

In case of the CLR, that is the CIL. Python, PHP, Clojure, F# and others happen to have compilers to this intermediate language.

Code gen bindings limit the accessibility to the language it has been done for, or makes it at least harder, depending on the implementation.

I think wasm for such an intermediate format seems more sensible, as said.

But having at least the dotnet languages with one binding, would have been nice too.

Anyway :man_shrugging:t2:

45

Our goal was never to include a .NET runtime in Defold but to specifically add support for C# as an additional extension language. We decided to do this via Native AOT compilation of C# code, not via JIT compilation of IL code.

The idea is also that the C# to Defold engine bindings should be generated from a C API. The Zig bindings should also be generated from the same C API.

3 Likes
46

So does this mean it’s possible for us to use our own language in Defold?

I have my own language, which is a modified version of Lua based (only changed some reserved words to make them more semantic, and I also changed the file extension to one I prefer) Basically, it’s still Lua—just a sort of dialect.

47

It’s not that easy.

  1. Our scripting language is lua, and we use LuaJIT 2.1 beta or Lua 5.1 as backend. If you build your libraries with a dialect, you still still run most of the engine on regular Lua.

  2. We don’t have easily replace’able toolchains to add other languages. We have support for Teal, and others have added support for Haxe and Typescript. But it’s still a bit advanced tooling around that.

  3. Native languages != scripting. Having native support still requires the engine to be built, and it requires a custom Extender server. It’s doable, but you need to do a lot of heavy lifting yourself.

1 Like
48

Ahh.., alright, I understand. Thank you for your response, Mathias.

49

F# supports Native AOT and can consume any C# API that doesn’t use dynamic, so I don’t know why we would purposefully ignore the possibility for its support.

I don’t know about Clojure.NET, but ignoring these languages just because we don’t think its worth investing 10 minutes, seems a bit off.

Godot did that mistake, Defold does have the chance to avoid that mistake.
Also, its seems kinda obvious that simply including a WASM API, would solve all that questions, as it gives instant access to a ton of languages, for the maintenance of one.

50

My $0.02 for what it’s worth: language is far less important than typed or untyped. Any professional programmer can pick up a language relatively quickly, but writing thousands of lines of untyped code in any language carries long-term maintenance costs.

When I tried teal with defold initially, I hit a few bumps (like incorrect defold API definitions). I’m curious to try again now that defold and Lua are both more familiar.

Edit: Teal says “Use a specific version for development and release to avoid breaking changes,” but the asset only has a single release, from May 2024.

Edit 2: “Defold Buddy” (which provides type annotations to teal for the defold API) is not yet compatible with defold 1.10.x: Project initializer creates invalid type annotations · Issue #9 · mikatuo/vscode-defold-buddy · GitHub

51

Implementing a new language does not take 10 minutes.
Also, we’d need to maintain it forever.
Just implementing C# took more than half a year. Sure, a lot of that work was layig foundations for more languages, but it just shows that the contact area is fairly large.

So far, we haven’t seen any usage of the C# feature either, so it’s not certain it will survive in the long run.

So far, we’re focusing on our C/C++ api, which seems more used.

6 Likes
52

I think there is a misunderstanding.
It seems like you implemented C# very uncommon.
Thanks for answering all my questions.

2 Likes
53

Depends on what you mean. We decided to use Native AOT as that fits with our goal of adding another compiled extension language. We worked with early access and beta versions of .NET 8.0. The amount of issues and bugs we ran into and reported to the .NET team was significant. We had to wait for fixes several times during the development process. Support for iOS and Android is still experimental. It will be better in .NET 9.0.

How would you have implemented C# in Defold?

2 Likes
54

(side note, we already support DotNet 9, which is their latest supported version)

4 Likes
55

I still don’t quite understand how you did it.
And I also don’t know for sure if that excludes other .NET languages.

And as I understand it, do you compile C# to binary, and then call that library?
I have never encountered such an implementation, so I can’t say much about it.

I am only a bit tired of the C# ecosystem constantly doing its best to somehow bend itself backwards,
to avoid compatibility with F# and Clojure.NET due to infighting at Microsoft.

Anyway, if it were my job to implement the language support in a game engine, I think that the Truffle AST from GraalVM would be at the very top of my list.

That would give us support for Ruby, Javascript, Python, Clojure, Kotlin, Java, Scala, LLVM, WASM, and more.

And if that weren’t an option, I would look straight to some WebAssembly runtime, like WAMR or wasmer, to support at least the WebAssembly languages.

I would give the people the most options possible, and still focus on one of these languages to write documentation for, leaving the rest to the community.

Few things have been so annoying to me as trying to add support for custom languages into a game engine.

And if you are someone who wants to write e.g., functional code, or something else that is not quite mainstream, you are on your own. :frowning:

I understand it’s hard to implement support for a scripting language in a game engine.

So just do it once, and support as much as possible with it. :man_shrugging:

56

Correct. We compile the C# code to a static library and link it with the engine, just like any other extension code written in C, C++ and ObjC.

I don’t have any experience with Graal. I see that GraalVM can build to static libraries. Do you know what kind of binary overhead to expect from a basic compilation of a single class file? And how much does Truffle add? What about runtime performance? Also, does it support arm and x86? What about consoles? Web?

1 Like
57

So, to give you a simple rundown:

Truffle is a framework written in Java that allows you to implement a “Truffle language”.

That is usually done by implementing the AST of the target language, but can also be the bytecode.
You implement your language as an AST interpreter using Truffle’s APIs.

That language gets the AoT compiler (called Native Image) the JIT compiler, a profiler, a debugger, profile-guided-optimization, and so on: All languages do use the same set of those tools, as they all become “one language”. You can also debug in the browser, with the Chrome tools.

So you can combine Ruby code with Javascript code, all that works over the Polyglot API.
The languages get ultimately compiled to the same AST, called Truffle AST.

Runtime performance is excellent, the Ruby implementation is orders of magnitude faster than any other implementation, and all the other languages are very quick and quite small as well.

Of course not on the level of pure C and C++ code, but it can be optimized for speed or size, and I know the wasm export (which is also a potential output format, next to hosting it) is quite small by now.

They are constantly improving this, and are currently working a lot on the wasm support.

It is otherwise supported on ARM64 for macOS and Linux, and x86-64 on Windows, macOS, and Linux.

Console support is something I am not so sure about:
Those use these common architectures, and I guess Xbox should use Windows, but idk about the others.
Probably not supported, but I assume they could support wasm?

Again, the reality that Graal can would tie in neatly with a simple wasm extension, as this reduces the footprint and the challenge of implementation: People could just use the GraalVM to implement the language, and ultimately use the wasm runtime to connect to Defold.

They could also use wasm directly via Go, Grain, or other languages that compile to it.
It’s also inherently compatible with the web.

I think that could be the ideal solution, since it means we keep almost all the benefits of Graal, while the use of a wasm runtime as extension host for plugins is more common, and probably better standardized.

A couple of programming IDEs, like Lapce and Zed do that already, and also the Microsoft Flight Simulator on Xbox, as a couple of examples. :slight_smile:

I can research that deeper, if you want.

EDIT: I just found this repo that contains a bindings creator for languages that adhere to a certain standard. That is currently C#, C/C++, Go, Python, Rust, and Javascript. So these languages could be implemented with relatively less effort.

1.) wasmtime is the best WIT/component model compatible runtime
2.) wit-bindgen and other tools help to reduce the implementation effort for bindings

This would, so far as I understand, complicate shipping the game on the web.

58

It seems that the minimal size of an hello world native image built with graalvm is around 7 Mb, which makes it a complete non-starter for web.

1 Like
59

Not having previous knowledge of, I carried out a short internet search for info on Truffle and GraalVM (without AI:-) and conclude:

Truffle is very slow and requires high memory and CPU usage.
Defold games would need to run in a JVM.

Is this correct?

60

You can strip that, then you get to around 1mb.
And yeah, shipping a runtime comes with its cost.