the second video in the series. We’ll record more, if you think we have to clarify something.
4 Likes
Johan,
Hello again. After running the live update successfully, I’m stopped by a non technical issue now. Hopes you can show me a hit and thanks again.
According to your reply contents, a resource has to be included or excluded when bundle the application. There is no chance to do a live update for an non excluded resource. The developer has to decide whether the element is excluded(able to update) or not(unable to update) when bundle the application.
In my case, the game download about 33MB data with 285 files from the local file server(the same PC where the game client runs) in about 16 seconds.
live_update_download.zip (10.1 KB): download file log
It will be rather slow when a remote file server with CDN is used in the production environment. The bottleneck here is the single file download mechanism.
I have a workaround to reduce the size of the downloaded files by following steps:
- All resource including texture, font, animation, sound ans etc with large size are bundled in the application.
- The gui, script and lua codes are marked as excluded and will use the resource which are already in the bundled application.
- If the UI layout or the script/code logic is changed, the live update for changed contents are suitable.
- If the resource in the bundled application has to be updated, create a new excluded resource and download via the live update and then replace the old one. Of course, the component which use the old resource shall also be updated to use the new excluded resource.
The downloading size will be significantly reduced(90%~) and the left parts are almost small files(might be less then 10 KB for each). But the file count is reduced little and is still too large. The first live update downloading costs too much time.
Is it suitable to do a special first time live update in the user code? Like downloading a zip file(generated in the bundle) from the server and unpackage it when there is no live update manifest before(the engine provide an API to check if there is any live update contents in the local storage). In the later live update, just use the single file download.
It sounds like doing that many separate http requests adds up to overhead rather that having big resources.
If you know what resources you will download at first-launch, perhaps you can try serving them in a single (or several) zip files and reading the zipped content yourself? There are a bunch of lua libs for reading data from zipped files, but I haven’t tried them personally.
3 Likes
Hi Johan,
I had tried to using a lua lib to read data from a zip file these days. I can get the raw data from the zip, but the data is incorrect due to the char set. The live update files in the zip(generated in bundle windows application) have both ANSI and windows-1251 format. Would you please take a look at that?
Hello! Can you give more information on you setup? What lua-lib are you using? What platform (win/mac/linux)?
platform: windows
file: da39a3ee5e6b4b0d3255bfef95601890afd80709
view in HEX:
origin content:
data content in the zip file(change to windows-1251 manually):
data content in the zip file(ANSI):
This link is the description for the structure of a PKZip file:
https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html
Here are the zip files(I just unpack the defold zip file and pack all files again without any compress, using store only):
defold.resourcepack_4251621488239601207.zip (6.2 MB)
defold_live_update_test - 副本.zip (7.1 MB)
defold_live_update_test - 副本.zip (7.1 MB)
My question is that why this file da39a3ee5e6b4b0d3255bfef95601890afd80709 using window-1251? You may take a look at the origin zip file and there are several files using this special char set.
2 Likes
Great repro, thanks. I’ll have a look at this next week.
Our build pipeline basically writes all resources to file, we then use Javas zip-util to produce the zip file. Might be some hitch along the way that causes the weird char sets. Sounds strange though that char sets differ between resources, I would have expected them to all have the same.
3 Likes
Hi Johan,
I’m not sure about this description in the manual.
As observered, the engine start and the loaded manifest will never changed even called the resource.store_manifest API. The new manifest is downloaded and stored successfully and there will be no missing resources returned by the collectionproxy.missing_resources API. When restart the application, the engine will load the updated manifest and download the missing resources. Is it possible to make the new manifest working after the resource.store_manifest finished without an additional restart? Or is there any limitation for this?
Thanks
1 Like
Not currently no, since the manifest loading is part of bootstrapping the engine. You can restart the engine in script though: https://www.defold.com/ref/sys/#reboot:arg1-arg2-arg3-arg4-arg5-arg6
1 Like