Is there such thing as "too many modules"?

[theHandgun]

Hi all,

I have 2 questions.

1. I expect to have 500-1000 lightweight module files to store dialogues, items, NPC data and more. Should I expect to get into trouble with too many modules, is there a hard limit on how many modules can be loaded?

2. I am using string concatenation such as require("modules.items"..item.id) which prevents module file to be loaded by Defold engine. I used a workaround which is to use require() function to a “module_registrar” module with full names of every module that I will use with string concatenation. Is there a better way to do this?

[Chung_Xa]

Is it possible in Lua? In many languages, we are not allowed to do so

[theHandgun]

I am not sure which part you are talking about so I will answer for both.

I am currently storing a few debug items, dialogues and NPC Lua modules and it works like a charm. Using conditions and effects of these things in the storage level makes it very convenient to add complicated stuff if needed.

For string concatenation, it also works like a charm. My only worry was memory usage but after some benchmarking, I found that loading all these module references as functions instead of data tables saves some memory, although, I need to spend more time making sure.

Here is an example file:

return function()
	local M = require("modules.items.base")()
	
	M.name = "Apple"
	M.type = "Food"
	M.desc = "Delicious synthetic apple, union approved!"
	M.effect_txt = "Health +5"
	M.img = ""
	M.weight = 1
	M.is_stackable = true
	M.actions = {
		{name = "eat", func = function(actor, inventory)


		end},
		
		{name = "drop", func = function(actor, inventory)


		end},
	}

	return M
end

My questions are mainly for the engine limitations but of course, I am open to suggestions of any kind.

[Chung_Xa]

Why do you end up on this solution? Not sure of there’s any limitations but I know that you will take a lot of time when building bundles

[theHandgun]

Since I am making an RPG, there will be a lot of intertwined mechanics, it wouldn’t be so wild for my game to have items with effects such as, when you eat it and have intelligence above 8, a door opens up and some dialogue options become available.

These complications are best solved in item data rather than through some item manager scripts in my view. It also provides better extendability and management.

Do you have an approximation on bundle times with many modules, is there a reference point that I can look into to decide if it’s going to take too long that I should consider another method?

[NaakkaDev]

Have you seen this? Where to put data: Lua vs JSON

[theHandgun]

This is a very valuable resource, thanks for sharing.

So it looks like storing data in Lua will result in 15-20% more memory consumption compared to JSON and it’s going to be a bit worse on mobile. These trade-off’s are alright to me.

I wonder what the overhead memory consumption of every Lua module is. Considering that I will have many small scripts. If the overhead memory spike during the collection load is too high, I would have to merge bunch of modules into bigger sized files to reduce the module count.

[britzl]

@theHandgun

• Should I expect to get into trouble with too many modules, is there a hard limit on how many modules can be loaded?

There is probably some very high limit. Every required module will end up in the package.loaded table, keyed by module name. A Lua table probably has some very high cap on number of keys, but you’d probably run out of memory before hitting that limit.

@theHandgun

Is there a better way to do this?

You could package all those tiny Lua modules as resources using the Custom Resources functionality, load them using sys.load_resource() and parse them using Lua loadstring().

[theHandgun]

Thank you for your help, I’ll look into sys.load_resource().

[playmedusa]

Sorry for side stepping a bit, may I ask about this approach? I’m also making a game with a (much lower) number of items, with their own effect. Simplest example would be kind of ranged weapons. For instance a gun or a flamethrower.

Both shoot but their behaviour is very different. The gun applies damage immediately while the flamethrower creates a number of “fire actors” in a line (to create some kind of wall of fire).

I do have a file with item descriptions, but while I considered adding also functions, I ended up leaving there just data. Then how each weapon acts is handled in the ranged_attack_command. Which would grow a lot if I had many weapons, which I don’t like too much…

I guess you prefer these definition files with data and functions?

[gameoffate2018]

For a few of my games I used a module system combined with inheritance. I come from an Object-Oriented Programming background, so it makes the most sense to my mind. I don’t like copy / pasting functions since that makes fixing bugs and adding functionality harder.

It’s worked well for me in the games I’ve made. However, I’ve never tried with 500-1,000 items.

Here’s the approach I used in Rogue Royale Reloaded (50-60 items):

At the high level I had an item module with a name, ID, and some functions common across all items (can spawn as loot, use item, load, save, etc).

From there I created several sub-modules for ammo types, weapons, clothing, etc. I could change or add functions that would be used by all members of that class. Clothing would have a health value, weapons would override the ‘use item’ function to shoot. Etc.

Finally I would create a module for the item that had the description, ID, and any other functions that would be specifically needed by that item.

In the end the structure looked something like this:

• item
  – Clothing
  — Armor
  ---- Knight Armor, Suit, Space Suit, etc
  — Helmet
  — Etc

I did keep a singleton file with all of the items in a table so I quickly grab items by ID.

Hope this helps!

[playmedusa]

Sure it helps! Sounds indeed closer to OOP. Thank you a lot for the insight.

[theHandgun]

I am doing something very similiar, happy to see that it worked out for someone else out there.

@playmedusa

Sorry for side stepping a bit, may I ask about this approach?

My system is just like the system of @gameoffate2018 so it should clear it out. Although, with each action (equip, use, drink throw etc.), I am returning an instruction set (heal, equip, increase_stats, spawn item etc.) to the item script so that the world can be affected through these items as well.

A side note, I don’t expect to have 500-1000 items but module files. I also hold NPC, dialogue and entity data as module files so they all combine to 500-1000 numbers. Not very important for the topic but I just wanted to prevent any confusions.