Parsing of require() statements could be improved (DEF-2522) (SOLVED)

Hi there,

I know that the Defold engine builds a list of all necessary lua scripts by parsing require() statements out of the code. However, there are a few scenarios that I think the engine could try to support. Scenarios that would work in regular lua, but that don’t work in Defold. Here they are:

  1. Importing a lua package

Consider a lua package consisting of a collection of lua modules like this:

  • mypackage
  • init.lua
  • foo.lua
  • bar.lua

In regular lua, the contents of mypackage/init.lua can be required like this:

require(‘mypackage’)

But Defold’s require parser looks for a file called “mypackage.lua” and can’t find it. It demands that the statement be written as:

require(‘mypackage.init’)

  1. Importing and indexing in a single statement

Sometimes I run into a situation where I have a module that contains a singleton-like instance of an object. Suppose I have a file called “model.lua” that contains the following:

local M = {}
M.model = init_my_model_or_whatever()
return M

In consuming code, I want to get access to this model instance, so my options are:

local model = require(‘model’)
model = model.model

But that’s a tad confusing. Another option is:

local libmodel = require(‘model’)
local model = libmodel.model

But again, that’s a tiny bit more work than I’d like. What I really want to do is:

local model = require(‘model’).model

But this isn’t recognized by the Defold require parser.

I’m posting this in the Bugs category, because I think that the rule of thumb should probably be “if it’s syntactically correct in lua, then it’s syntactically correct in Defold”. Even if you don’t feel that it’s necessary to add support for these cases, finding a way to provide more descriptive error messages to the developer might be nice. Tracking down the problem in the second use case took me quite some time because the require parser simply misses the statement all together, and the lua file can’t be found at runtime. :slight_smile:

1 Like

Thanks. Yes, this is a known issue, also reported here: Incorrect parsing of `require()` expression (DEF-2522) (SOLVED)

Created an issue so that we can solve this once and for all: DEF-2522

2 Likes

Ah, sorry, somehow I missed that other post. Thanks!

1 Like

No worries. I’ll try to fix this when I’m back from GDC.

Fixed in Defold 1.2.101

1 Like

Just FYI: I’m using editor 2, v1.2.102, and part 1 of my initial post doesn’t seem to be fixed. I still have to include .init on the end of my pacakge imports.

You mention that this is possible in plain Lua but as far as I know that’s not really true. I’ve tried in Lua 5.2, Lua 5.1 and LuaJIT and I’m not able to import an entire folder of Lua modules in a single require() call. Maybe you’ve tested with a version of Lua that has a custom package loader?

Actually, I was partially wrong about part 1. I just checked the default package.path in lua 5.1 and it contains the following paths (on separate lines for readability):

.\?.lua
C:\Program Files (x86)\Lua\5.1\lua\?.lua
C:\Program Files (x86)\Lua\5.1\lua\?\init.lua
C:\Program Files (x86)\Lua\5.1\?.lua
C:\Program Files (x86)\Lua\5.1\?\init.lua
C:\Program Files (x86)\Lua\5.1\lua\?.luac

So you can see that if I try to require('foo'), lua will attempt to find the following files:

.\foo.lua
C:\Program Files (x86)\Lua\5.1\lua\foo.lua
C:\Program Files (x86)\Lua\5.1\lua\foo\init.lua
C:\Program Files (x86)\Lua\5.1\foo.lua
C:\Program Files (x86)\Lua\5.1\foo\init.lua
C:\Program Files (x86)\Lua\5.1\lua\foo.luac

So, you’re right that it won’t find a local package like ./foo/init.lua, but it will find a standard lua package called C:\Program Files (x86)\Lua\5.1\lua\foo\init.lua (if one existed).

I can’t for the life of me imagine why lua doesn’t include .\?\init.lua as the second pattern in the package.path array, since it leads to this weird inconsistency.

In any case, what Defold does is consistent with what Lua does, so I’m happy :slight_smile:

Sorry for the confusion.

2 Likes

No worries! And just to be clear: In Defold we replace the default contents of package.loaders with a single loader that loads files from within the game archive. This means that the Lua version shipped with Defold will never ever be able to require modules that are located elsewhere on disk, unless you define your own loader.

1 Like