LiveUpdate philosophy

#1

We have been developing a major project for over a year. And in it we are using LiveUpdate. And because of this, we constantly encounter problems along the way.

However, this approach has many limitations (this is my old topic about it):

  1. if you build the same project on different PCs, its LU will not work with the old loader.
  2. if at least one byte has been changed in the loader, the LU will not work with the old loader.
  3. if the project version has changed, the LU will not work with the old loader (although the version has no effect on the content).
  4. if you want to update the loader to the new one, it is impossible to do this through the LU.
  5. when downloading files from LU it is impossible to know their size to show the progress bar.
  6. and other things we have not found yet.

We tried splitting our project into a loader (as abstracted from the game code as possible) and the game itself. This solution helped to get a more or less workable version (“more or less” because it only works on version 172 of the engine).

Perhaps the thing is that the LiveUpdate has a name that sort of speaks of a live update - i.e. the ability to update content, to make it updated and fresh.
And we try to use LiveUpdate to update our content - to deliver some parts of it independently.
So we come to the point where the LU still requires a complete update of the entire project from the store.

What’s the sense of the LU then?

Okay, I understand that this is probably a pointless question, so I want to ask if it’s possible to get some kind of feature (let’s call it AutoUpdate) which will have the following properties:

  1. any game collection can be excluded from the bundle and deposited somewhere on the internet.
  2. the game will try to download the collection when it detects it.
  3. this will not depend on the PC where the collection was built, neither the version of the loader, nor the engine version (but it will depend on the format - if the format has changed, you need to update the bootloader).
  4. the game’s start collection (i.e. “in game” loader, not engine) can also be updated by the same way.

How feasible is this wish?

5 Likes

LiveUpdate clarifications
#2

Good questions! I’m going to spend time tomorrow to update myself on the current state of the LiveUpdate feature so that I can give you a good answer.

6 Likes

#3

That just seems like a bug to me. The bundling should rely on determinic checksums (e.g. Sha256).

if at least one byte has been changed in the loader, the LU will not work with the old loader.

This has to do with the checksum verification. The intention was to to avoid tampering with the resources, to make sure noone attempts a man-in-the-middle attack.
I guess an option to remove all signature checking is possible, but not sure if that’s desired?

when downloading files from LU it is impossible to know their size to show the progress bar.

That seems like a separate request, to get callbacks to the http.request() for download status.
Not sure if it’s possible to use a “HEAD” request before, to figure out the size, but since we cannot get the progress currently, it’s probably not so useful for showing a progress bar.

What’s the sense of the LU then?

As you say, the name implies the intent.
The idea is to be able to update content while the app is live, in order to avoid the long app submission process.

However, this feature is still in it’s first (almost at least) iteration.
Currently it basically only supports separating large game content from the the the main app bundle.
Sadly, we’ve never gotten to investigate how to update the LiveUpdate feature with more of the required functionality to actually update the “live game”.

this will not depend on the PC where the collection was built, neither the version of the loader, nor the engine version (but it will depend on the format - if the format has changed, you need to update the bootloader).

Adding a checksum for the file format isn’t something we have, and it’s going to be tricky, since we’re adding resource extension api.
However, it might be needed for our next version of LiveUpdate.

  1. the game’s start collection (i.e. “in game” loader, not engine) can also be updated by the same way.

I’m curious, how often does this collection really change? What types of changes are we talking about?

Engine version

We have the support for supporting different engine versions in the manifests.
Each time the liveupdate bundle is created, the “liveupdate.supported-versions” is baked into the manifest. These versions are checked when the the manifest signature is checked.

liveupdate.settings:

[liveupdate]
supported-versions = "1.2.180","1.2.181"

I haven’t tried this feature myself, so I’m not sure what caveats there are.

MVP2
To sum it up, what we originally intended to support (it’s still the current main idea), is to download new manifests. These manifests will work as before, referencing other resources and their checksums. The difficulty lies in how to verify this manifest, since the engine version loading it might not be the engine version that generated it with regards to file format. It’s vital that the engine doesn’t try to load something it doesn’t support.

While writing this, I also feel like “engine version” I mention above should almost work with respect to updating the game with new manifests. But we need to investigate it to find out more about what the current state is, and what needs to be done to make it easier to use.

4 Likes

#4

Yes, it should be possible to do a HEAD first, but as you say, since we do not have a progress function it is of little use.

You can at least have a progress bar that increments as each file is downloaded.

We could leave this up to the developer to deal with. Let them secure the application the best they can.

2 Likes

#5

We are very careful about this, so it would be convenient to have the responsibility on our side.

We thought exactly the same.
This is exactly the kind of functionality we need!

In our case, it turned out to be small changes that happened all the time. It was nothing global - many cosmetic adjustments. However, it is important to note that the loading logic of the rest of the game remained largely unchanged.
So the old downloader might well have downloaded the new game correctly. But this turned out to be impossible. And that was sad.

This is what we have done now. And it works well, as we have more than 1,500 files.

2 Likes

#6

I have started testing with a small sample project and actually came across a crash when downloading a new manifest, rebooting and loading missing resources. I’m surprised that no one has reported this before and that we haven’t caught it either. Anyway, it is reported on GitHub and we will look into it soon.

0 Likes

#7

Do other engines implement this? Given the other thread about Apple’s review process, this kind of feature makes me think they’ll flag Defold games w/ LiveUpdate more often.

I don’t have any plans on releasing for Apple, but sometimes Google likes to copy them.

0 Likes

#8

Yes, but maybe not all engines. And a lot of games have different kinds of live-ops systems built in to be able to show new content when running different kinds of campaigns etc or to tweak gameplay settings or patch levels after release. I know King users this extensively in most of their games. They do not have a system where they can replace anything at any time so it’s not as powerful as LiveUpdate.

We have been discussing a way to allow developers to completely exclude the LiveUpdate system from their games using an app manifest, but it’s not possible to do this yet.

1 Like