Liveupdate strange behavior

#1

We found strange behavior in LiveUpdate.

Situation (example):

  1. we have the sources of some project.
  2. we make a build and a LiveUpdate on PC1 and upload it to the Internet.
  3. everything works!
  4. then we do the same build on PC2 (but we don’t do the LiveUpdate because the sources are the same).
  5. new build doesn’t work!

What’s wrong with this situation?
Each of our developers has to generate his own LiveUpdate each time to test the build, despite the fact that the sources of the LiveUpdate are the same (we didn’t change collections inside LU).

Is this correct behavior?

0 Likes
#2

function resource.store_manifest() pass status == resource.LIVEUPDATE_BUNDLED_RESOURCE_MISMATCH

0 Likes
#3

but resources didn’t mismatch - there are absolutely identical!

0 Likes
#4

When you say “make a build” on PC1 you mean bundle to .app or .exe? And the same on PC2?

Are you also downloading the manifest that was generated by PC1 and use resource.store_manifest() on PC2 before checking for missing resources?

And what happens when “new build doesn’t work”? What kind of error are you getting?

1 Like
#5

Building with same settings ( debug/release, texture compression on/off, etc )

  1. On PC1 build apk1 + exe1 + liveupdate
  2. Unzip and upload LU on server
  3. On PC2 build apk2 + exe2 (without LU)
    Finally on device apk1 with exe1 work well :
  4. anyway load new manifest
  5. success store new manifest
  6. reboot app
  7. load missing resources
  8. store them
    But apk2 and exe2 can’t store new manifest with error code LIVEUPDATE_BUNDLED_RESOURCE_MISMATCH ( -8 )
    If don’t download new manifest cannot find some missing files
0 Likes
#6

Thanks for the additional info! @JCash what are your thoughts on this?

1 Like
#7

I’ll describe the problem one more time.

Let’s assume that we have absolutely identical sources of the same project (via GIT) on two different computers, PC1 and PC2.
We build the project (APK) on PC1 and upload LiveUpdate to the Internet.
The APK is working fine, as expected.
Then we build APK for the same project on PC2, but this time without LiveUpdate.
The APK of this assembly does not work (because it does not like the LiveUpdate manifest).

That’s the problem.
It seems that LiveUpdate is tied to a specific PC. Specifically, the one he was building to.

1 Like
#8

This is very important for us.
In the our new project, we aim to bring most of the project to LiveUpdate. Ideally, the entire project.
LiveUpdate’s current behavior will cause problems along this way.

1 Like
#9

Me and @JCash talked about your problem today and agreed to increase the priority on LiveUpdate related problems in general.

Created: https://github.com/defold/defold/issues/5033

1 Like
#10

Unfortunately, so far I haven’t been able to reproduce it.
I attached a small test project to the issue that worked for me.
Please see if that works for you? Or supply a small test project I can try out.

0 Likes
#11

Please check are you reproduce all the steps I wrote above (8 steps).

1 Like
#12

I did not have a complex enough example (with a manifest and rebooting). And I do get some issue of storing the manifest currently, after I’ve created the liveupdate content on my OSX machine, and trying to load it from a windows executable. In my log it mentions:

DEBUG:SCRIPT: verifying and storing manifest liveupdate.game.dmanifest
ERROR:DLIB: Decrypt: rsa_alt_decrypt_public_wrap failed: -17028
ERROR:GAMESYS: Manifest verification failed. Manifest was not stored.
DEBUG:SCRIPT: Failed to store manifest, status: -8

You mention that you get LIVEUPDATE_BUNDLED_RESOURCE_MISMATCH too, did you get any other info in your log file?

Also, what platforms are PC1 and PC2 running?

1 Like
#13

We run Defold editor on Win10.
Yeah, we get a line in the log file:

DEBUG:SCRIPT: Failed to store manifest, status: -8

1 Like
#14

Problem is that LU and all versions of the application, even test versions, must be created on the only one same computer. This makes it difficult for the team to work. If the next test assembly of the application is created on another computer, then the published LU can no longer save that manifest. But all builds ( new and old; android/ios/win ) with all versions work fine on all devices when those was build on machine who created and publish manifest.
Thanks!

1 Like
#15

We’re investigating this. And we do acknowledge that it makes it hard for a team to work efficiently.

2 Likes
#16

Although I’m not 100% sure this is the case you have, I did find an issue with the way we bundle the “game.public.der”. If it doesn’t find the cert, it silently (!) generates a new one for you.

And, in my case, the liveupdate.settings had absolute paths in it, which ofc don’t work well on a different machine:

[liveupdate]
zip-filepath = /Users/mawe/work/bundles
publickey = /Users/mawe/work/projects/users/mawe/Issue-5033-live-update-manifest/public.der
privatekey = /Users/mawe/work/projects/users/mawe/Issue-5033-live-update-manifest/private.der
mode = Zip
supported-versions = "1.2.172"

When I changed the paths to be project relative, they worked:

[liveupdate]
zip-filepath = /Users/mawe/work/bundles
publickey = /public.der
privatekey = /private.der
mode = Zip
supported-versions = "1.2.172"

Could this be the issue for you?

In either case, I’ll make some changes as to how that works, so that we’ll get a build error if a non-existent path is specified.

2 Likes
#17

Yes! Our certificates are stored in the root folder of the project and the path was specified relative. And we tried, on the contrary, to specify the full absolute path to each key and it works !! Thanks a lot for the tip!

2 Likes
#18

Great that we found the root cause!

I’m surprised that you couldn’t use the relative path, since that’s what I did :confused:

Anyways, I’ll continue working on better error reporting.

2 Likes
#19

We have now achieved that we have one version of the LiveUpdate built on PC1, and this LiveUpdate works correctly with absolutely identical builds built on PC1 and PC2.
BUT! If we change at least one line of code in the build on PC2, this build stops working with LiveUpdate - says “can’t store new manifest: code -8”.

P.S. this line of code presents outside of LiveUpdate

0 Likes
#20

I’ve fixed a few things that would make the hashes differ between platforms (coming to dev branch soon). And I think I now have the scenario which you currently have. I changed a script file on my PC2, and then the downloaded manifest won’t match that file anymore.

ERROR:RESOURCE: Resource ‘/main/main.scriptc’ is expected to be in the bundle was not found. Resource was modified between publishing the bundle and publishing the manifest?
ERROR:GAMESYS: Manifest verification failed. Manifest was not stored.
DEBUG:SCRIPT: Failed to store manifest, status: -8

This currently is as it’s been designed though, since normally, you bundle your new project version, and then upload it.

I would need to know more about your intended workflow to be able to better understand the situation.

1 Like