Don't fetch extensions if they are up to date

Feature requests
#1

I’d like to increase Defold open speed by not redownloading extensions that are up to date.
For instance most of the time they are hosted at GitHub which provides API to get latest commit info or release info. The extension URL can be parsed to check if master is requested or a release is requested and by invoking certain API we can check if there were any changes since the last fetch.

For master we can check commit.sha
https://api.github.com/repos/lerg/extension-adjust/branches/master

For release we can check published_at
https://api.github.com/repos/lerg/extension-adjust/releases/tags/1.0

8 Likes
#2

This would be super great for loading and fetching to speed it up. You should make an issue for this here too

#3

Afaik, we already check if the library is the latest one, by checking the ETag?
@vlaaad or @mats.gisselson might provide more insight.

1 Like
#4

It doesn’t seem to be functional. Or is just super slow.

1 Like
#5

Yes, on slow internet connection the opening window stalls for quite a while on updating dependencies.

1 Like
#6

I also vaguely recall some issue regarding ETag and how GitHub serves content. Perhaps that’s part of the issue.

3 Likes
#7

I also think an alternative solution to this problem would be to allow the user to skip fetching if they want to. Extensions don’t change all that often.

Also, yes, I can confirm that, at least on our end, Defold downloads the 60MB-ish FMOD extension every time we open the project.

5 Likes
#8

Investigated a bit. It does look like there is indeed an issue with ETags and GitHub currently. The file is always downloaded from most GitHub library dependencies. Maybe there’s another way we can do the request, but we need to investigate further.

9 Likes
#9

Since most extensions tend to be hosted on github, maybe if “github.com” is detected in the URL, you could do custom Github-specific stuff to check for cache busting, like commit hash, for example.

3 Likes
#10

Can confirm that dependencies are always loaded for me too, even if there are no updates. There should probably be an option to skip fetching the libraries if the versions can’t be detected reliably.

luasec for example is 21-22MB in size, which gets downloaded every single time the editor is newly opened. Even with a fairly fast internet connection, this still slows down the editor start-up, and that’s just for 1 extension. Sometimes when downloading from github is unusually slow, I have to wait more than 10 minutes and then the editor just fails to open completely and will just auto-close for some reason.

Fow now, I had to copy local versions of my dependencies so the editor won’t have issues.

3 Likes
#11

You are not alone :stuck_out_tongue:



5 Likes
#12

We will look into this this week.

4 Likes