Moddable games with Defold

[Alex_8BitSkull]

This is just a general thread to collect thoughts on how to make Defold games moddable. I am putting in some legwork to try and outline the fundamentals as I understand them, but I would love to hear your thoughts on anything I might have missed or misunderstood. Certainly by the end of the posts I have more questions than answers.

What do you need to make a game moddable? Well, you need tools and editors for people to create content - and I am going to ignore that for now… What I am more curious about at this point, is how to make Defold incorporate and respond to the content generated by users.

The first hurdle, then, is how to make Defold see the user generated content. Referring to this thread, I find that Lua has an I/O library which should be helpful in loading files. Lua doesn’t have a concept of a file system with folders, however. A dirty fix would be to load a specific index file with paths to all other required content, but a better solution might be Defold-LFS - a Defold native extension written by @britzl for the LuaFileSystem library (“a portable way to access the underlying directory structure and file attributes”).

So if we can load files, what kinds of files and what do we do with them?

The simplest, yet very powerful, would be loading some sort of data (narrative, character stats, enemy characteristics - you name it). You could load a JSON file and parse it with the in-built Defold JSON API or the CJSON extension (not sure what the relative benefits are).

What about images? A good answer here appears to be ImageLoader written by @sergey.lerg, which allows loading of images external to the game. This is fine for static images, but I am not aware of how you would go about dynamically creating (or even replacing existing) atlas animations.

Getting even more complicated: Can we load and execute external scripts? Do we even want to - if it’s potentially a security risk? This post certainly makes it seem possible to do so with bundled files, perhaps you can do the same with external files using the file system solutions mentioned above?

What about sounds and music? I’m not sure. This is perhaps more related to a broader question touched upon in the secion on images: Are there any of Defold’s components that could be loaded externally? Modified by externally loaded content? Can we change a default three-frame animation of a running man to a five-frame animation of a horse?

[Jerakin]

This is not an answer to your questions just a brief overview of my experience.

I kind of did this for my App, which is a app for a Pokemon homebrew to D&D (super nerdy I know. I wrote a blog post about the app a while ago).

What I did was to create a python app which they (meaning the user of the mobile app) can use to put in all the information manually needed for a Fakemon, and believe me there is a lot. Before it is saved the app validates it against some json schemes then packs it up. With it you can create a new Pokemon or edit an already existing one. The user would then send the created package to me, I would validate it (because of Google and Apple user agreements/restrictions) and then upload it to the package library, which is simply a github repository. It’s important to note that if I could chose freely I would not have liked to do it this way. I would have liked them to be able to have a zip file on their phone that they could read in (relevant to “The first hurdle”). But there is no extension for finding files on Android/iOS so that wasn’t possible, this was the only reliable way to do it. “Luckily” I only have about 500 DAU and not that many people do these packages so I do not have to validate and look through that much.

The mobile app have a “Fakemon Package” setting where you can specify which package you want to download (I put in a restriction so you can only use one, then I don’t have to resolve what happens if they use two conflicting packages). If you select one the app will download it and then restart (to reload with the new data). After a restart the data is there to be used. Relevant code here.

The package consists of a json file and (presumable) images of the Fakemon. When a fakemon package is enabled/selected it checks if there is a custom path defined if there is it loads the image with file:readand uses gui.new_texture to show it in the app. Relevant code here.

I have many excuses why my code is so bad here are a few: I am not a programmer by trade, the programming I know is self taught, this was only meant to be a small project, it’s only a hobby project and I get no money from it, I have very much a “just make it work” mentality with the project.

[dapetcu21]

Yes, you definitely can load scripts. Yes, it’s a security risk, unless you make sure to sandbox the function you’re executing using setfenv(). An environment in Lua is the “global table” it has access to. You should probably copy references to math, vmath, print and other stuff from the global table to your newly created environment, but you might want to leave out more dangerous things like io and require() (though you might want to re-implement that for the user)

[Alex_8BitSkull]

@Jerakin and @dapetcu21 - thank you both for your very informative posts!

I kind of did this for my App, which is a app for a Pokemon homebrew to D&D (super nerdy I know. I wrote a blog post about the app a while ago).

I think you’re in good company here. I like D&D myself and have heard of Pokemon variants, but never tried one myself. Carry on!

[sergey.lerg]

For sprites you’d have to implement them on your own out of plain png files. TexturePacker can help.
For scripts you can create a native extension with a completely separate Lua instance in it and run your user’s code in there in a sandbox.
So it’s possible, but hard.

[britzl]

This is just a general thread to collect thoughts on how to make Defold games moddable

I like it!

how to make Defold see the user generated content

I guess this depends a lot on how the mod content is created and added to the game. If mods are downloaded from somewhere you’d could keep a list of downloaded mods and that would be the way you see the content. But using Defold-LFS and loading files from a mods folder probably makes the most sense.

You could load a JSON file and parse it

Yep, JSON is fairly easy to edit and very easy to load and parse into something that can be used by Lua.

What about images?

This is fine for static images, but I am not aware of how you would go about dynamically creating (or even replacing existing) atlas animations

Yes, static images are quite easy to load. Animations not so much (unless you handle animations yourself and create your own flipbook playback system).

Getting even more complicated: Can we load and execute external scripts?

Not complicated. This is what many games does. But I think it will be easier if you design your game for modding from the start. And your game should define a mod API to help modders. Here are a few ways to load and run Lua code: https://github.com/britzl/publicexamples/blob/master/examples/load_lua/load_lua/load_lua.script

What about sounds and music?

FMOD (https://defold.com/assets/fmod/), OpenAL (https://defold.com/assets/openal/) and Mod Player (https://defold.com/assets/modplayer/) can load sounds on the fly.

[Alex_8BitSkull]

I guess this depends a lot on how the mod content is created and added to the game. If mods are downloaded from somewhere you’d could keep a list of downloaded mods and that would be the way you see the content. But using Defold-LFS and loading files from a mods folder probably makes the most sense.

Would this ‘mods’ folder best be placed in the same folder as the game itself, or something like appdata/roaming (like DefSave does it, for example)? Are there any particular considerations for either choice, or does it not matter?

Yes, static images are quite easy to load. Animations not so much (unless you handle animations yourself and create your own flipbook playback system).

Just for clarity’s sake - there’s no way to load external game object components (like atlases), right? I assume no.

Not complicated. This is what many games does. But I think it will be easier if you design your game for modding from the start. And your game should define a mod API to help modders. Here are a few ways to load and run Lua code: https://github.com/britzl/publicexamples/blob/master/examples/load_lua/load_lua/load_lua.script

Thanks for the link! What are your thoughts on sandboxing, as mentioned by dapetcu21 and sergey.lerg? I’ll do my own research on things like setfenv(), just wondering if you have any thoughts on it, like if there are limitations or special things to consider, etc.

Re: “it will be easier if you design your game for modding from the start” - absolutely agreed. I am researching not for Fates of Ort, but for my next project

[britzl]

Just for clarity’s sake - there’s no way to load external game object components (like atlases), right?

Correct. It is not possible.

What are your thoughts on sandboxing, as mentioned by dapetcu21 and sergey.lerg

Do it! I totally agree with their conclusions.

I am researching not for Fates of Ort, but for my next project

Ohh, nice!

[Mathias_Westerdahl]

What we did in Just Cause 2, was to leave our “dropzone” folder mounted, even after release, and that meant that any existing files could be overridden that way, by simply dropping in content with the correct format and name. It might not solve all use cases, but probably a few. (And the modding community could really go to town with that, with very little help from us developers.)
Adding a function to mount a folder should be fairly easy.

[Alex_8BitSkull]

This sounds very much like something I will end up doing!