Luajit 2.1 memory allocator issue

gianmichele · January 5, 2022, 8:33am

While lurking around love2d, I noticed that they mention a luajit 2.1 memory allocation issue for Arm64.

Are you guys aware of this? There’s also a bug report here:

github.com/LuaJIT/LuaJIT

Increase available allocation range in mcode_alloc()

opened 01:10PM - 25 Feb 17 UTC

akopytov

enhancement 2.1

This is an improvement request related to #282, #283, #284 and mmap() scalabilit…y issues. mcode_alloc() uses half the range defined by `LJ_TARGET_RANGE` to make all allocated blocks mutually reachable by relative jumps. However, for multi-threaded applications with separate JIT states per thread, that requirement is unnecessary. The only required constraint is to make static assembler code reachable from all allocated blocks, which is ensured in mcode_alloc() by calculating `target` based on the `vm_exit_handler()` address. I propose to change mcode_alloc() so that instead of all execution threads using the same allocation range of `[target - range/2; target + range / 2]` (where `range` is the jump range defined by the architecture), make some threads use the `[target - range; target]` range, and other ones use `[target; target + range]`. I have an experimental patch assigning either one of the two possible ranges to JIT states based on a PRNG. The patch essentially doubles the available allocation pool for mcode_alloc() and helps reduce mmap() scalability issues for multi-threaded applications.

Mathias_Westerdahl · January 5, 2022, 8:38am

We don’t use the JIT part of LuaJIT on several platforms that don’t allow it/support it, e.g. iOS and consoles.
We do however use the LuaJIT library as its interpreter is faster than the vanilla 5.1 one.

That said, no I didn’t know about that particular issue.
In general, I’d say the library works well, and if an issue arises from it, we’ll have to deal with it.

gianmichele · January 5, 2022, 8:51am

I’ve run some tests on my M1 Pro and I’m not seeing that issue since Defold is still not native on Apple Silicon.

Mathias_Westerdahl · January 5, 2022, 8:55am

On macOS, we also set the base pointer address iirc, in order to make sure that the LuaJIT address range would fit. (At least we used to, and it should be possible again I think)