Is there any chance Defold will ever support metamethods such as __len, __pairs, __ipairs and __le & __lt (for mixed types). As I understand it, LuaJIT must be built with the flag -DLUAJIT_ENABLE_LUA52COMPAT. Will we, as users of Defold, ever be able to take advantage of these great LuaJIT features?
1 Like
I think there are still platforms like HTML5 where LuaJIT cannot be used and they’re using plain Lua 5.1 instead, so even if DLUAJIT_ENABLE_LUA52COMPAT would be enabled, you won’t be able to use those features everywhere.
2 Likes
While that may be true, Defold is already using LuaJIT by default on platforms that can use it. Defold currently offers Some Lua 5.2 features already through LuaJIT (such as goto). Even though there are platforms that can’t use LuaJIT, Defold has decided to use LuaJIT where applicable.
So, I’m asking, for those platforms that are currently able to use LuaJIT (which already ships with Defold), would it be possible to flip that build switch (DLUAJIT_ENABLE_LUA52COMPAT) to enable the the other Lua 5.2 features such as those listed above?
@JCash actually touched upon the LuaJIT lib today as part of another task. I’m not sure if it is as easy as flipping a switch though. Mathias do to have some input on this?
My main concern is if users were to write Lua code that wouldn’t work on all platforms.
As for “end users”, i.e those that write code only intended for their game, they can ofc choose to use features above 5.1 when using LuaJIT. Problems will ofc arise when sharing code with others (e.g. on forum or via assets).
Ofc, we already have this situation, that the interpreters support a different set of functionality.
And we do mention that in the documentation, along with that the recommended version is 5.1.
So, I guess, as long as we still support 5.1, it might be possible to add the extra functionality to LuaJIT, at the users discretion.
A much bigger concern is if we then break existing code for our users. When looking at the LuaJIT documentation, the “unlikely to break things” features are enabled by default, but LUAJIT_ENABLE_LUA52COMPAT is left as an option. Question is, how likely is it that our users code will break, from one release to another? Such a caveat makes me much less eager to go down that route. I think some research is needed to answer that question.
As it stands, I don’t dismiss the idea, I just need more info about the ramifications of upgrading the library.
2 Likes
Britzl and Mathias, thanks for the quick reponses.
@britzl, I’m certain you’re correct about it not being a simple matter of flipping a switch since you’re operating in a production environment and I think Mathias outlined some of those details well. I guess I meant that perhaps there wouldn’t be a ton work for you guys to do in order to get it online since it should build with that switch change.
@Mathias_Westerdahl , I understand your concern about current code possibly not working correctly after enabling added features. I wonder if doing a sandbox on the Defold source and running unit tests on it could answer that question. Mike Pall may even be of help in answering some of the questions regarding compatability and risk to the Defold stable release.
If I understand Mike’s comment about the COMPAT switch correctly, it looks like his main concern regarding breaking things is the _ENV variable (and how it applies to the global environment). Although, it should be noted that _G is left in for backward compatability. His secondary concern seems to be that the COMPAT switch only partially supports lua 5.2 which means anyone depending on using the full breadth of Lua 5.2 features would be out of luck and they would also have to be careful not to assume the existence of Lua 5.2 functionality.
Additionally, if it turns out that you cannot be aboslutely certain whether or not there may be adverse effects (which may be mitigated), you could always offer the Lua 5.2 compat build as experimental and as a “use at your own risk” type of thing. You could even tag projects built with the experimental versions so people opening them would know that they may not be compatible with the stable release of Defold.
Wow! Relevant indeed. I wonder if that might be even better than LuaJIT.
Regarding the _ENV_ comment, it outlined why not even more 5.2 functionality was added, since that would definitely break the ABI compatibility. The section still stands, as something that may break other functionality.
Asking the author might be good, providing he has the time to answer.
As for moonjit, the adoption is still very low (from Nov 2019), although it might be an option in the future. However, they don’t seem to have a clear mission statement, so I’m not sure what actual version they aim to support. (5.1 or otherwise)
1 Like
Yeah, I’d say that unit testing would be in order to check for those breaking points in current code.
Regarding MoonJIT, it looks like they are currently already supporting much of Lua 5.2 and some of Lua 5.3.
It appears that most of the Lua community that participated in that Google code survey use Lua 5.3 out of the box. The next up is some form of LuaJIT. The fact that people are using Lua 5.3 could be an indication of them moving away from LuaJIT. Especially since most of the those people using LuaJIT are using the experimental version. Likely due to LuaJIT not supporting modern Lua (and the fact that the LuaJIT project is all but dead at this point), I would speculate that people may be moving on from LuaJIT trying to get Lua 5.2+ functionality. As Lua moves forward, LuaJIT keeps falling farther behind. Eventually, LuaJIT will be outdated to the point of being unuseable. That is, unless someone picks up where Mike left off and starts development again.
This is tricky, because a game engine would be misstepping to go back to plain Lua considering that a JIT version is many, many times faster than standard Lua. But, how do you get that speed and keep current if the JIT version is falling out of date as Lua moves forward? More and more users will be asking for modern lua functionality as time goes on. And as time passes the Lua version continues to increment but the JIT in the engine stays the same . Well, who knows, maybe the Defold team could be the folks to take over LuaJIT development from Mike. He’s been out the development phase for a long while now, looking for someone to take over the project.
No no. Remember that Lua is used in so many other places than games and in places where performance isn’t as important but where newer language features might be.
I see your point there. Although, I would say that the fact that most LuaJIT users are using the experimental version indicates that they are looking for more features.
I assume this is 2.1-beta3? It’s a beta so I wouldn’t call it experimental. And it has been in beta for a long time so it is quite stable by now. I believe another reason for choosing 2.1 over 2.0 is the arm-64 support which can only be found in the 2.1 branch*
*=I’m not entirely sure about this but I believe it to be the case.
Whatever your long-term decision on LuaJIT 2.1 in Defold might be, is there any chance you could release an experimental version of Defold with the LuaJIT 2.1 branch in it for those if us who’d like to use those features?
No, that chance is slim I’m afraid. We’d have to build that specific version of LuaJIT for all platforms and create a way for you to select this version of Lua when you bundle your game. Perhaps it could be done using the native extension build server. In any case it requires a lot of time and effort spent on something which quite frankly would mean little gain. We’d rather spend the time elsewhere.
2 Likes