Module mess - how do you structure your projects?

[Alex_8BitSkull]

Ever since I understood and started making more use of modules my projects have improved drastically in terms of organisation and readability.

However, I invariably create lots of modules and multiple modules end up wanting to make use of each other. For example, in my latest project I have a “crew” module and a “ship” module. At some point they will each want to call a function in the other. That doesn’t work, of course. Two modules can’t require each other or else the space-time continuum is shattered.

You can try to avoid the modules needing anything from each other, but I’m finding that hard to do.

You can subordinate one to the other, allowing one to require the other but not vice versa. However, I’m now finding myself duplicating main module variables into the submodule (e.g. in the main module having something like: submodule.somevar = M.somevar). That feels messy.

How do you handle this? If you don’t have to, how did you prevent it from being an issue in the first place?

[WhiteBoxDev]

It sounds like you’re using too many modules (gasp that’s not supposed to be possible!)

In my mind, a module should be created when you notice a significant amount of code that could be generalized. If it can’t be generalized, then it should probably remain in its original .script file instead.

Another way of thinking about it is to ask yourself “could I make this chunk of code into an external library that’s usable by others?” Again this comes back to the need for generalization. If you’re placing Ship-specific code (for example) into a module, and that code depends on code in other modules, then it likely shouldn’t be in a module at all.

I see a lot of confusion arising from developers moving too much of their code into .lua files simply to avoid a small amount of boilerplate code. This can cause unintended complexities and confusion. If I had to choose between some boilerplate vs complexity, I would choose boilerplate.

Of course, one exception to the generalization rule is something like raw data tables.

Edit 2024: Module mess - how do you structure your projects? - #21 by WhiteBoxDev

[Alex_8BitSkull]

It sounds like you’re using too many modules (gasp that’s not supposed to be possible!)

You’re probably right!

Of course, one exception to the generalization rule is something like raw data tables.

I do have a lot of these, but it’s not the sole purpose of the modules.

You’ve given me some stuff to think about. I think I’m overusing modules to the point of it being counterproductive.

Part of the reason, I think, is caused by another bad (?) habit of mine. I like structuring things to work synchronously, which is very easy to achieve with a module (crew.get_crew()) versus, for example, the asynchronous method of sending a message to a .script (msg.post(“crew_controller”, “get_crew”)) and waiting for a response.

Your points about boilerplate are also reasonable, but it has taken me a long time to move away from repeating myself so getting back into the habit will be tricky. However, I’m sure what you’re advocating is a sensible and healthy use of boilerplate whereas my old practices would probably be described as unhealthy.

[WhiteBoxDev]

@Alex_8BitSkull

I like structuring things to work synchronously, which is very easy to achieve with a module (crew.get_crew()) versus, for example, the asynchronous method of sending a message to a .script (msg.post(“crew_controller”, “get_crew”)) and waiting for a response.

This can usually be solved by using go.property() instead of keeping data under self.<var>. Although I suppose maybe not if crew is a table, since we can’t store tables in go.property().

If crew is a table, maybe try initializing the table in crew.script (or wherever it’s housed). Then, on initialization of ship.script (or wherever you need access to the crew list), call msg.post(“crew_controller”, “get_crew”) and save the result as a local table (self.crew_ref for example). Since Lua doesn’t perform deep copies of tables by default, you should now have access to the memory address of the crew table in both scripts. You won’t have to use get_crew anymore, unless the memory address of the table is changed in crew.script.

Edit 2024: Module mess - how do you structure your projects? - #21 by WhiteBoxDev

[Alex_8BitSkull]

That’s a neat trick and makes perfect sense. It’s not the only reason I’m overusing modules but it’s one piece of the puzzle!

[ross.grams]

Hmm. In your crew/ship situation, I would not expect the two modules to actually depend on each other. Obviously the crew needs to interact with the ship, but presumably those interactions would work no matter what kind of ship it was, right? (Or have sensible failure checking…“well duh, this is a robot ship with no living quarters, you can’t house crew in it.”) In more specific terms, I would expect the crew functions to be passed in everything they need. If they need to interact with the ship, then the function will be sent a ship object, or whatever ship data it needs.

Especially with modules, try to keep your functions as “pure” as possible, like math functions. A function should only do what it says on the tin.

[d954mas]

I use one module that require all game modules.
world.lua

Link to world passed in constructor, or world after require another module, set world var in module.

module.world = self

When ship need crew.
ship.world.crew
When crew need ship.
crew.world.ship

[Alex_8BitSkull]

Hmm. In your crew/ship situation, I would not expect the two modules to actually depend on each other.

I felt tempted to “defend” my design here by explaining in more depth how my game works, but there’s no point. It boils down to:

Especially with modules, try to keep your functions as “pure” as possible, like math functions. A function should only do what it says on the tin.

And:

In my mind, a module should be created when you notice a significant amount of code that could be generalized. If it can’t be generalized, then it should probably remain in its original .script file instead.

What I’ve started doing is, rather than moving generalised code into modules, is group (an excessive amount of) code into modules based on the part of the game (crew, ships, enemies, etc) it relates to. What I’m getting from your replies is that my specific code should reside in a .script file and only the generalised parts should be in a module. The example of a math module is a good one.

I need to think about whether a big refactor is in order for my current project, but certainly it’s something I should keep in mind from the beginning for my next one.

[ross.grams]

I felt tempted to “defend” my design here by explaining in more depth how my game works, but there’s no point.

Haha, but I can’t think of a more fun way to waste time than by having a lengthy debate on game/code design!

It is hard to be sure when the Return-On-Investment of refactoring becomes worth it. I usually do it when I realize that I’m avoiding doing something because the code is too annoying to work with. In my case refactoring sometimes means “dumbing down” and just copy-pasting more stuff, rather than trying to fit everything into a big, perfect system.

[Alex_8BitSkull]

I felt tempted to “defend” my design here by explaining in more depth how my game works, but there’s no point.

Haha, but I can’t think of a more fun way to waste time than by having a lengthy debate on game/code design!

To be clear, I meant “there’s no point” not as “because you wouldn’t understand” but as “I’m probably wrong anyway”.

It is hard to be sure when the Return-On-Investment of refactoring becomes worth it. I usually do it when I realize that I’m avoiding doing something because the code is too annoying to work with. In my case refactoring sometimes means “dumbing down” and just copy-pasting more stuff, rather than trying to fit everything into a big, perfect system.

Broadly speaking I’m actually quite happy with how it’s working right now. It’s just when I run into the case of wanting to cross-require that I realise I’m doing something naughty. It’s probably not worth a big refactor right now… Let’s let future Alex worry about that (when it’s inevitably going to be an even bigger mess!).

[gianmichele]

My modules tend to be self contained and specialized but don’t know anything about data. They take an input and return a result.
All the data lives in other lua files, but it’s only data, no functions.

[britzl]

My modules tend to be self contained and specialized but don’t know anything about data. They take an input and return a result

When I started out with Lua I created a lot of stateful Lua modules. My modules had state and functions mixed. This felt natural as I came from an OO background. Nowadays I have unlearnt some of my bad habits and I always try to keep the data separate from modules. The data is passed to a function, the function makes some kind of operation on the data and optionally returns some data back.

[yeqwep]

Do you think this module is good?

[britzl]

Is the purpose of the param.lua module to store different kinds of parameters used by several scripts?

I need to better understand what you are proposing. Let’s start with the basic example where the cat.script does a msg.post to the dog.script. What is the purpose of the message? And in the same example using param.lua the cat and dog script access param.pos? Is one of them setting and the other reading the value?

[yeqwep]

The purpose of the param.lua module is to manage all the variables.
In this example, the dog or GUI reacts to the cat’s position, items, and button presses.
I have failed many times with the left side method.
However, I have also failed with the right side method.
I want to achieve object orientation…

[britzl]

I have failed many times with the left side method.
However, I have also failed with the right side method.

Perhaps what you need is a combination? You can keep the application state in param.lua and use message passing to notify listeners/interested parties when something changes.

In this example, the dog or GUI reacts to the cat’s position, items, and button presses.

• The cat starts moving and lets the dog know through a message
• The cat has updated it’s inventory and sends a message to tell the dog that the inventory has changed
• The cat presses a button and sends a message to the dog that the button has been pressed

[yeqwep]

Thank you very much.
I need to understand more message passing.

[partypooper]

@britzl

• The cat starts moving and lets the dog know through a message
• The cat has updated it’s inventory and sends a message to tell the dog that the inventory has changed
• The cat presses a button and sends a message to the dog that the button has been pressed

This examples are good for one direction messages.
But what if dog wants to know cat’s name.
In OO style it would be almost as simple as cat.name which is synchronous and doesn’t split the logic in half, with messages you can get it only with (or am I missing something?)

-- somewhere in the dog.script
function half_of_the_logic(self. params)
   -- some logic here, then it needs cat's name to continue
    msg.post(cat_url, "say_ur_name_say_ur_name")
end

function on_message(self, message_id, message)
    if message_id == hash("hi_my_name_is") then
        -- another half of the logic is here. How to trace it? How to wait for it?
    end
end

-- somewhere in the cat.script
function on_message(self, message_id, message, sender)
    if message_id == hash("say_ur_name_say_ur_name") then
        msg.post(sender, "hi_my_name_is", { name = "Slim Shady" })
    end
end

Could someone help me to wrap my head around this behavior? I can provide real-world example that I want to achieve too, if you desire to help with it too.

[britzl]

I think you would benefit from this technical writeup to help you think about a way to structure your code using Lua modules.

Technical Journal

I suppose I’ll update this thread with posts that are related to Defold. October 7, 2024 Scripts or Modules? It’s an Architecture Decision, Moreso Than a Data Decision This was a big one. I hope you find it valuable. Have you ever found yourself struggling to decide what data goes where, whether some logic might be better suited to live in a script versus a module, which scripts should be responsible for managing which modules, or some other organizational problem that inevitably results i…

[partypooper]

The post with most likes in this thread

@WhiteBoxDev

In my mind, a module should be created when you notice a significant amount of code that could be generalized. If it can’t be generalized, then it should probably remain in its original .script file instead.

Your link to @Vow journal, that has this

Scripts are drivers. Their sole purpose is to provide channels of communication between the engine and the user through lifecycle functions. Keep them lightweight.

Don’t they contradict each other?
I do understand lua modules and that you need to move some data (is it possible to move all data out of script? It still has some data in itself like sprite size etc), move some logic to modules too. I just don’t understand what exactly I should move. I’m not sure if it is my gaps of knowledge of some game development or programming overall, or defold is really missing some documentation about this. I have searched on a free time this week and I haven’t found any good resource that explains how to correctly split scripts and modules (of course there is no one right way, but at least some explanation in manual or examples in examples section would be good). Maybe it’s not related to game engine itself and more related to some programming paradigms, but when engine did this separation it would be good to provide some documentation on this part.

I have real-world example, so maybe someone could point me what they would do:

1. Let’s say we a crate created with crate.go, which is some kind of grid based collection of items/resources.
2. We have an inventory from which we can drag-and-drop items to the crate and from the crate.
3. We have some assembly line constantly “adding” some items to the crate too.
4. So we have 1 inventory, we have few crates created with a factory, we have various kind of items and we have one or more (not relevant to example) assembly lines.
5. Now we dragging item from inventory into one of the crates that is constantly updating it’s state from the attached assembly line and we while holding item under some cell of it should receive information could we put this item into this cell in current situation (like is the crate already full, does it accept that amount of items, does it accept this kind of item etc etc).

How would one do this in defold?
Because in love2d it is easy, and I really not sure what to put where in defold. And your provided article doesn’t answer my question (I totally understand that it could be my lack of knowledge).