HTML5 and live update

Trying to integrate liveupdate in an HTML5 project and having problems. Already switched Content-Type to apllication/octet-stream
What else could be the culprit?

[Error] ERROR:GAMEOBJECT: Could not instantiate game object from prototype /main/Prefabs/Menu/Pages/MainMenu/mainMenuPage_generated_0.goc.
[Log] WARNING:RESOURCE: Unable to create resource: /main/Prefabs/Menu/Pages/MainMenu/mainMenuPage.collectionc: FORMAT_ERROR (dmloader.js, line 504)
[Error] ERROR:GAMESYS: The collection /main/Prefabs/Menu/Pages/MainMenu/mainMenuPage.collectionc could not be loaded.
1 Like

Did the HTML5 project work before adding live update features?

Without excluding a collection - yes.

This type of error message rings a bell.

Could it be that the file isn’t actually what you wanted it to be?
I.e. is it the same on the receiving end as it was on the server side?

I took a single file for inspection. Downloading with wget gives the same file - so the hosting is ok.
In the game I printed the response of http.request() when getting this file as hex and compared with the original - also the same content.
So yes, resources seem to be downloading just fine.

1 Like

I have a vague idea of what’s going on. I think resource.store_resource() is not thread/coroutine safe. When downloading many resources it might corrupt the live update archive file.

And when it’s corrupted it fails to create game objects from it. Even when changing the code to a more consequent downloading process and reloading the page. Clearing website data (html5) or clearing local files (desktop) helps and Defold creates uncorrupted archive file anew.

1 Like

I’m currently not sure what happens in your case to make the archive corrupted, but we shouldn’t have any threads enabled on HTML5 builds.

That said, we recently did an update to the Live Update system to allow for downloading the entire archive at once. Perhaps that’s something you can use as a workaround?

Sorry, I didn’t mean actual OS threads. I meant Lua concurrency. When for instance invoking two http.requests() and they both call resource.store_resource() in the callback. I think in this case the resource archive might get corrupted.

I will try storing entire archive.

Well, we shouldn’t have any concurrency issue in this case, since we don’t use threads on HTML5. :thinking:

But store_resource() is async, right? Maybe calling store_resource() again before the first call hasn’t finished somehow messes things up? Since both invocations are working on a single archive file?

Both requests will end up in a live update queue, and that queue is processed one item at a time.

I mean, there shouldn’t be any concurrency related issues, but it seems from your case there is an issue there somewhere.

2 Likes

Still having issues with downloading live update files. Sometimes works, sometimes doesn’t.

I tried implementing downloading the entire archive, but having problems as well. After downloading the archive the resource.is_using_liveupdate_data() is not returning true, even after a delay. When I run the game, it downloads the archive, reboots, downloads the archive, reboots and does it a few more time before resource.is_using_liveupdate_data() is finally true.

Is it possible to use the downloaded archive without restarting the game?

Here is my code.

-- This can be anything, but you should keep the platform bundles apart
local ZIP_FILENAME = 'resources.zip'

local APP_SAVE_DIR = 'game_liveupdate'

function init(self)
	self.base_url = 'http://localhost:8000/'

	print("LiveUpdate Archive Loader: is_using_liveupdate_data:", resource.is_using_liveupdate_data())
end

local function store_archive_cb(self, path, status)
	if status == true then
		print("Successfully stored live update archive!", path)
		--
		timer.delay(4, false, function()
			if resource.is_using_liveupdate_data() then
				msg.post(self.callback_listener, 'liveupdatearchiveloaded')
			else
				sys.reboot()
			end
		end)
	else
		print("Failed to store live update archive, ", path)
		-- remove the path
	end
end

function on_message(self, message_id, message, sender)
	if message_id == hash("attempt_download_archive") then
		self.callback_listener = sender
		if resource.is_using_liveupdate_data() then
			msg.post(self.callback_listener, 'liveupdatearchiveloaded')
		else
			local url = self.base_url .. ZIP_FILENAME
			local path = sys.get_save_file(APP_SAVE_DIR, ZIP_FILENAME)
			local options = {
				path = path,        -- a temporary file on disc. will be removed upon successful liveupdate storage
				ignore_cache = true -- we don't want to store a (potentially large) duplicate in our http cache
			}
			print("Downloading", url)
			http.request(url, "GET", function(self, id, response)
				if (response.status == 304 or response.status == 200) and not response.error then
					-- register the path to the live update system
					resource.store_archive(response.path, store_archive_cb)
				else
					print("Error when downloading", url, "to", path, ":", response.status, response.error)
				end
			end, nil, nil, options)
		end
	end
end

After more experiments I can confirm it works better if I download Defold generated resource pack ZIP file, extract it using pure Lua zzlib, and call store_resource() on each file without getting missing resources from proxies.

But why are you not using resource.store_archive() to download and use an entire zip file of exclude content? Why unpack it manually?

Because it doesn’t work or I can’t understand how to use it properly. When I install the archive using resource.store_archive(), my collections are still reporting missing resources. I have to restart the game using sys.reboot() several times for it to take effect.

1 Like

I see. Could you please create an issue on GH and share a minimal project reproducing the problem?

3 Likes

It should only require to be rebooted once, to load the new archive. The callback is called when the reference is written. After that, it should be ok to reboot.

I wonder if it’s related to HTML’s file system, that the file isn’t actually ready to be read until much later? Perhaps you can check for the files existance in the callback? (and/or the update function)

What’s the behavior on mac/win for you?

1 Like

I will try to make a test project after the holidays, thanks.

It’s the same for HTML5 and macOS for me. After store_archive’s callback is fired with success state, I don’t see liveupdate.arcd file created.

If rebooting is required for store_archive() then it’s still not an option since store_resource() doesn’t require rebooting. Thanks for clearing this up.

Nor should you. When using the archive version, we read directly from the .zip file you specified.

(although internal names, they’re “liveupdate.ref.tmp” (before reboot) and “liveupdate.ref” (after reboot))

2 Likes