As @sicher said, the ability to load and use a new manifest and update the game with new content that way is being developed as we speak. It is a big task though so no definitive ETA I’m afraid. But we really aim to finish it soon since it adds a powerful feature to LiveUpdate.
2 Likes
Thanks @sicher and @Johan_Beck-Noren for the quick replay. It is possible to bundle the new game content locally and generate a “manifest” by myself and use resource.store_resource() to update the game? You may know the apple store review often take weeks so we are highly dependent on the live update for the online activities of the festival.
As it is today you cannot change or update the manifest that is bundled with the game. This means that you cannot, with liveupdate, add or change resources that were not listed in the original bundled manifest. This new feature will let you do just that.
More info on the manifest: https://www.defold.com/manuals/live-update/#_the_manifest
2 Likes
perhaps I can resurrect the old Live Update topic. The feature is shipped by now - you can create and ship new content to your live games without updating the client.
Did anybody actually try the feature? If not, what stopped you? Any concerns/questions?
@Johan_Beck-Noren and I are recording a video next week, and are thinking about additional angles to cover.
6 Likes
Hello @jakob.pogulis,
After reading the live update descriptions, there are some unclear points for me. Would you please take a look and help to give some feedback?
, thanks for your kind help.
- live update settings
There are two modes in the editor and one is the “Zip” model in which I get the zip for the excluded collection proxy in my project. After decompressing the file, there are the liveupdate manifest file and lots of hash named resource files.
I didn’t get the point for this zip mode. What’s the difference to the AWS mode? Can the engine do the check and start updating the manifest file from a local zip file? Or the AWS is the only place to hold this liveupdate manifest file currently? This is important for me. If yes, I can do a complete live update test offline using this zip mode.
- Updating the manifest with Live update
There are a version mechanism to detect whether the bundle has been changed in the manual page:
Starting the engine for the first time after a manifest has been stored will create a bundle identifier file bundle.ver next to the manifest. This is used to detect whether the bundle has changed since the manifest was stored, for example after a full app store update. If this is the case the stored manifest will be deleted from the filesystem and the newer bundled manifest will be used instead. This means that a full app store update will delete any previously stored manifest. Any existing Live update resources will however remain untouched.
How does the engine check this version? I assumed that the engine first takes a look at the project live update setting and knows that another live update manifest file(local file or url) is present. The engine decides whether the live update manifest file shall be download(or copy) according to the version. There is a tip about the fake code from the manual page.
If the zip mode in the question 1 does not work, can I use a local file server instead of the AWS for test? The only thing for the engine is to know how and where to get the file.
- storage of the live update and bundled manifest file
If the live update manifest file is dowloaded successfully, is it stored in another path without overriding the bundle one via the resource.store_manifest API? If not, do the engine get the correct file depending a special search path priority?
There is also a description for the verification.
When storing a new manifest the manifest data will be verified before it is actually written to disk. The verification consists of a number of checks:
Correct binary file format.
Supports the currently running engine version or any other supported version entry from the settings.
Cryptographic signature.
Signed using the same public-private key pair as the bundled manifest.
If the key and supported version is not filled in the project settings, what happens when call the resource.store_manifest API?
-
get the missing file of a collection proxy
How does this API collectionproxy.missing_resources(collectionproxy) calculate the missing file count? Check the hash of all resource in the bundle(live update if there is one)manifest file and try finding it? Will an changed resource with a new hash be treated as a missing file? -
download a single file
After getting the count of the files to be download, is it possible to get more information? Like the total and already downloaded size of the files which could be used to indicate the downloading progress? In the production environment, a single download might be failed due to the network satus. A retry mechanism as part of the error handling might be important. -
store the download file
Does the API resource.store_resource store the download file some where like the save-file path on different operating system? Or it just modify the bundled one?
If there a save path, when a excluded resource is updated for several times, will the old versions which do not match the latest live update manifest file be deleted automatically?
6 Likes
@Johan_Beck-Noren and I will record a video on Tuesday on Live Updates. We’ll go through your questions. And will see how we can improve the manual.
Thanks for your feedback!
1 Like
Great questions! I’ll try to answer them one by one.
The API for Live update does not make any assumptions on how or where to store your excluded resources. The Amazon AWS integration is there as a convenience if you prefer to server the resources on S3, but you are free to unpack the zipped resources and serve them anyway you like. I often spin up a local web server when testing stuff locally.
Basically, the version file writes the signature of the bundled manifest the first time the app starts. For every subsequent app start we check if the bundled manifest signature matches the signature written to file. If they don’t match it means the bundled app has changed (been updated). It is simply some logic to check if an app update has occurred and in that case to purge any locally store manifest file and fall back to the bundled manifest.
It is up to the user to supply a new manifest to the engine. We do not download a new manifest for you, you must host it yourself and download in (for example with http) and supply it with resource.store_manifest.
Yes, any stored manifest and liveupdate resources are persisted on local storage. The bundled manifest and resources resides inside the APK on Android for example, which we can’t really modify or overwrite in runtime. At engine start, we check the local storage for any manifest file and load that one instead of the bundled one.
The callback supplied to resource.store_manifest will be called with the status argument telling you want went wrong.
For every resource the collection needs to load, a lookup is done for the resource hash in the resource archive index. Any missing resource hash will be put in a list and returned. A modified resource will generate a different hash and therefore be treated as a different resource.
Currently no, the only available information about a missing resource is its hash. But since the user is responsible for hosting the missing resources it should be possible to for example query your server about the size of a file. The API is pretty lean in that regard and leaves the hosting-and-storing totally up to the user.
Yes, the actual resource data is stored on local storage, the same path as any liveupdate manifest. On desktop it is the app data folder, on device it is on local storage.
A modified resource will produce a new hash and will therefore be treated as a different resource. There is no state where we track different versions of the “same” resource, since they are not the same.
I have it in my TODO though to implement some kind of purging or unloading of liveupdate resources that might not be needed anymore or for some other reason needs to be removed from local storage. DEF-3114 tracks this feature.
11 Likes
Johan, thanks for your kind answers. The game now can do the live update using a local file server successfully.
The total game is about 80MB and the bootstrap(main) collection does not contain any game resource but only an excluded collection proxy which is the real entry of the game. In this entry collection, there are several collection proxies for different scenes. So the run time sequences for the first start are:
- Game start and get the version from the server including live update resource
- Download and store the live update manifest file
- Try to load the entry colloection and find that there are missing resource
- Download all missing resources and load the entry collection
- Try to load a scene colloection and find that there are missing resource
- Download all missing resources and load the scene collection
- Repeat 5~6
My target for the live update is to keep the ability to update most of the resources. Due to this limitaiton from the manual, I have to use this run time sequences and set all real game resources as excluded. So the first bundled game is an empty shell.
Is it possible to update a resource which is not marked as exclued? For example, the first bundled game contains a base version which holds most of the resources which are stable(like image, sound and 3D model, non source code resource). And it’s still able to do a live update for the resources in this bundled game using a priority mechanism(the resource in the save fold has high priority then the one in the bundled game). Then the first bundle application is a minimum playable game without additional downloading data. The size of the application might be 40MB which contains 80% usage of the game. Only the user who really triggers the scene needing a live update. Always downloading missing files one by one for a collection proxy really costs time. The purpose is to find a balance between the bundle size and live update size and try reduce the interruption.
Thanks a lot for your help.
3 Likes
Great that you are using this feature!
Im not sure I understand the question fully though.
It is not possible to modify the content in the bundle, so updating the bundled resources will not work. But I think it would be possible to do achieve something similar with Liveupdate by instead modifying what resources an excluded collectionproxy points to.
At build-time we create a resource graph mapping all dependencies between resources. This allows us to differ between resources that are only referenced in excluded collection proxies from those needed in the bundle. If for example a “model” resource if referenced in a bundled collection AND excluded collections, it will not be excluded since it is needed in the bundle.
- For the resources to be included in the bundle at all they need to be referenced by a collection somwhere, otherwise they are disregarded when bundling. You can then have excluded collection proxies which references these bundled resources if you like.
- These excluded collections could then be modified to contain/reference other resources. These resources along with the new manifest could then be pushed to the client, making the collection point to these new resources instead of the bundled ones.
The main point with the caveat is that any new manifest cannot contain changes to bundled resources. For example, you shipped with manifest A with a bootstrap collection “main.collection” that has hash A. You publish this to the app store. Later on you make some changes to “main.collection” (resulting in hash B) and publish manifest B with a ref. to new bootstrap collection to your client. When your game boots with manifest B it will try and find the resource with hash B in the bundle, but the shipped bundle does not have any hash B (the engine cannot find any bootstrap collection in this case, and will abort loading).
There is no versioning of resources. If you modify a resource it will generate a different hash and thus be treated as a new resource.
NOTE Liveupdate has collectionproxy-granularity so it’s only possible to exclude whole collections (and their resources), not individual resources. In theory you could of course have collections containing single resources, but it seems a bit impractical.
9 Likes
also here’s a video about Live Update and I am working on a new one to be uploaded tomorrow.
3 Likes
the second video in the series. We’ll record more, if you think we have to clarify something.
4 Likes
Johan,
Hello again. After running the live update successfully, I’m stopped by a non technical issue now. Hopes you can show me a hit and thanks again.
According to your reply contents, a resource has to be included or excluded when bundle the application. There is no chance to do a live update for an non excluded resource. The developer has to decide whether the element is excluded(able to update) or not(unable to update) when bundle the application.
In my case, the game download about 33MB data with 285 files from the local file server(the same PC where the game client runs) in about 16 seconds.
live_update_download.zip (10.1 KB): download file log
It will be rather slow when a remote file server with CDN is used in the production environment. The bottleneck here is the single file download mechanism.
I have a workaround to reduce the size of the downloaded files by following steps:
- All resource including texture, font, animation, sound ans etc with large size are bundled in the application.
- The gui, script and lua codes are marked as excluded and will use the resource which are already in the bundled application.
- If the UI layout or the script/code logic is changed, the live update for changed contents are suitable.
- If the resource in the bundled application has to be updated, create a new excluded resource and download via the live update and then replace the old one. Of course, the component which use the old resource shall also be updated to use the new excluded resource.
The downloading size will be significantly reduced(90%~) and the left parts are almost small files(might be less then 10 KB for each). But the file count is reduced little and is still too large. The first live update downloading costs too much time.
Is it suitable to do a special first time live update in the user code? Like downloading a zip file(generated in the bundle) from the server and unpackage it when there is no live update manifest before(the engine provide an API to check if there is any live update contents in the local storage). In the later live update, just use the single file download.
It sounds like doing that many separate http requests adds up to overhead rather that having big resources.
If you know what resources you will download at first-launch, perhaps you can try serving them in a single (or several) zip files and reading the zipped content yourself? There are a bunch of lua libs for reading data from zipped files, but I haven’t tried them personally.
3 Likes
Hi Johan,
I had tried to using a lua lib to read data from a zip file these days. I can get the raw data from the zip, but the data is incorrect due to the char set. The live update files in the zip(generated in bundle windows application) have both ANSI and windows-1251 format. Would you please take a look at that?
Hello! Can you give more information on you setup? What lua-lib are you using? What platform (win/mac/linux)?
platform: windows
file: da39a3ee5e6b4b0d3255bfef95601890afd80709
view in HEX:
origin content:
data content in the zip file(change to windows-1251 manually):
data content in the zip file(ANSI):
This link is the description for the structure of a PKZip file:
https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html
Here are the zip files(I just unpack the defold zip file and pack all files again without any compress, using store only):
defold.resourcepack_4251621488239601207.zip (6.2 MB)
defold_live_update_test - 副本.zip (7.1 MB)
defold_live_update_test - 副本.zip (7.1 MB)
My question is that why this file da39a3ee5e6b4b0d3255bfef95601890afd80709 using window-1251? You may take a look at the origin zip file and there are several files using this special char set.
2 Likes
Great repro, thanks. I’ll have a look at this next week.
Our build pipeline basically writes all resources to file, we then use Javas zip-util to produce the zip file. Might be some hitch along the way that causes the weird char sets. Sounds strange though that char sets differ between resources, I would have expected them to all have the same.
3 Likes
Hi Johan,
I’m not sure about this description in the manual.
As observered, the engine start and the loaded manifest will never changed even called the resource.store_manifest API. The new manifest is downloaded and stored successfully and there will be no missing resources returned by the collectionproxy.missing_resources API. When restart the application, the engine will load the updated manifest and download the missing resources. Is it possible to make the new manifest working after the resource.store_manifest finished without an additional restart? Or is there any limitation for this?
Thanks
1 Like
Not currently no, since the manifest loading is part of bootstrapping the engine. You can restart the engine in script though: https://www.defold.com/ref/sys/#reboot:arg1-arg2-arg3-arg4-arg5-arg6
1 Like