Reading files from "/res/common" folder with Emscripten (#4194)

I’m trying to read files from /res/common folder with c/c++ using fopen.

According to Emscripten docs, it is possible to use filesystem. But I couldn’t manage to load the files.
Is Defold preloading or embeding /res/ (Bundle Resources) folder content? Is it possible to use FILESYSTEM?

Seems like it isn’t Packaging res content… So how can we package the user content since we can’t use the --preload-file flag?

This is from the res/common of a native extension right? @JCash?

Nope. From Bundle Resources ‘res’ folder.

I’m trying to package resources with packager tool just for checking the if ‘-s FORCE_FILESYSTEM=1’ enabled in Defold or not.

This is the result.

I’m not sure but my guess is FILESYSTEM is not enabled. (I’m not sure if everything is correct or not.)
If you can enabled the FORCE_FILESYSTEM, or give us a flag for NE manifest then we can load files from NEs at least with packager tool

1 Like

For bundle_resources, they simply end up in the bundle next to the executable (or in a relative path if that’s what you specified), in this case your index.html.
After that, it’s up to the user to load it in their preferred way. Defold is not preloading these files.

Unfortunately, I’m not an expert on the HTML5 loading of files nor loading with Emscripten provided functionality. I’ll ping @sven who might know more.

2 Likes

You are correct, we don’t supply -s FORCE_FILESYSTEM=1 as far as I understand, maybe something we can add (or if users should be able to supply it in their app_manifest?).

As @Mathias_Westerdahl pointed out, bundle_resources for HTML5 should only output/store the resources alongside the index.html file. What are you trying to do? If you need to load some custom data could you add it to custom_resources instead? :thinking:

edit: Me and Mathias took a quick look at how our HTML5 version loads game data files (darc files etc). If you take a look at the dmloader.js file, you can see that it will download the files specified in archive_files.json (generated while bundling), then when finished it will map up these files with the emscripten function FS.createPreloadedFile (which will end up in the “emscripten filesystem”). Maybe you could hook into this somehow, if you really need it to be available through the “filesystem”?

It would be great actually.

I know. But what is the point of bundling files if we can’t load them directly from native extensions on HTML5… They can be anything; textures, videos or music files.

I’m loading mod music files (.xm and .mod format) from c/c++ with fopen. It is possible to use fopen (or similar approaches) on every platform except HTML5.
Simply adding them into the custom resources doesn’t solve the problem. It requires tons of extra work. I’m not planning to use custom resources for now.

Thank you

2 Likes

I see, we should perhaps change bundle_resources on HTML5 (or add a special option) so those files can be loaded through fopen as well… As a workaround for now, could you try loading/mounting them dynamically by modifying dmloader.js and using FS.createPreloadedFile?

I’m not 100% sure myself how I would do it, but I think modifying the archive_files.json might be an option, then the rest should be taken care of automatically (and you might not need to modify dmloader.js).

2 Likes

I’m not sure either but I’ll definitely try. Thank you.

2 Likes

@sven It works by adding files manually into archive_files.json. But not very user friendly solution for regular users.

2 Likes

I guess it can be automated by Defold with a simple checkbox. (Read file names and sizes from /res add it to json) And of course it can be done by using simple py script.

2 Likes

Agreed, I think we need have an easy way of doing it. Perhaps a special game.project setting for HTML5 builds, html5.archived_resources or similar.

I’m adding an issue for it: #4194

2 Likes

Great, thank you.

1 Like

And here is the result:

2 Likes

YESSSSS!!!

Thanks a lot for this extension!

2 Likes