Fetching engine and stuck

Hello, I was working with Defold and it was good to work but when I restarted Defold as soon as I close it and tried to build my project Defold got stuck showing ‘fetching engine’ and never show any result for more than 30 minutes. I tried to build again and it’s still same. So I checked my network if it’s the problem. My network is 90Mb/6Mb down/up so it doesn’t seem the problem. I checked if there is a new version of Defold but my one is already up to date. I tried to clean ‘.internal’ folder and build folder but it’s still same. Is it server’s overload because of Monday? This kind of situation is pretty embarrassing because I feel I can’t do anything to fix the problem. All my fate should be held by Defold server? I doubt if it’s good. I love Defold so much though.

Hmm, no the server is fine, and I don’t really see any anomalies in the graphs I have there.
Your internet speed seems fine, 30 minutes seems far too long.

Can you access https://build.defold.com ? (I assume so, or it should have produced an error earlier)

We have it in our roadmap to make the server accessible, so people who need it can have their own instance. ETA is still a bit uncertain, but “soon” is my best guess.

4 Likes

Yes I can access build.defold.com and got some text as below:

Extender
f79c250
2019-10-07 16:15:01

Sure I too think 30 mins is kind of abnormal so I’m figuring it out. First, I could know other tiny project like samples works without any problem. So I removed all dependencies from my project and then Defold builds it! And then I added one by one. I’m using below dependencies:

https://github.com/defold/extension-iap/archive/master.zip
https://github.com/defold/extension-facebook/archive/master.zip
https://github.com/britzl/defold-websocket/archive/master.zip
https://github.com/britzl/defold-luasocket/archive/master.zip
https://github.com/sonountaleban/defold-luasec/archive/master.zip
https://github.com/defold/extension-gps/archive/master.zip
https://github.com/defold/extension-firebase-core/archive/master.zip

When I add 3 dependencies related socket as below Defold was stuck again(without any other dependencies):

https://github.com/britzl/defold-websocket/archive/master.zip
https://github.com/britzl/defold-luasocket/archive/master.zip
https://github.com/sonountaleban/defold-luasec/archive/master.zip

So my conclusion is compiling native sources is very slow or having some problem.

By the way, why is Defold going to the service model not stand-alone like Unity? I’m one of those who think that it would be really nice if Defold comes out with a stand-alone model. I see a huge potentiality of Defold beyond Unity. But of course I know there is a big plan for Defold. It’s just my wish.

Yes, that’s because then the vanilla engine is already included in the editor/bob bundle on your local disc.

why is Defold going to the service model

  • We use native C++ (and Java/Objective-C) for all platforms, and those require SDK’s, and we cannot distribute that due to licensing reasons.

  • Size: not adding ~15GB of SDK’s to either the download size or consuming the space on your hard drive

  • Ease of use: Most importantly, all users can benefit form the extension system, without knowing how to deal with cross platform native C++ development. It is often a time consuming task.

Well, ofc it’s annoying when things doesn’t work, and one wishes for big changes, but I honestly believe in this model. And there are very few errors, and they’re mostly due to internet issues.

And, as I said, it’s in the plans to make the server available, we just don’t have an eta.

3 Likes

Add them one by one and bundle in between each one. See if you can pinpoint the problem.

We take care of cross platform compilation and bundling for all target platforms by having all of the iOS, Android, macOS, Windows, Linux and HTML5 SDKs installed and properly configured on our servers. You don’t have to worry about configuring a single thing when you add a dependency to your game. I bet many Defold users doesn’t even know that we build a custom engine for them when they bundle their project.

We provide you with a completely offline and stand alone experience if you do not use any native extensions.

Like Mathias writes we will make the native extension build server available to you soon so that you can set it up yourself and run a local instance in a docker container. It should hopefully happen in 4-6 weeks.

4 Likes

Wow. This is a super great news. It will be very useful especially when developing native extension with large libraries. :space_invader:

1 Like

This is exactly what I did to figure it out.

Then what’s the difference with Unity? Unity is a stand-alone but it can do what you said.

But without native it’s not practical. As you know professional game engineers should consider a lot of practical things like ads, in-app purchase, leader board, social, and etc. These are not done without native layer and it means somebody should support it. In my case, I tried to use 3rd party dependencies but had many problems to satisfy my criteria so I should make my own for all. And I realized it’s not easy job yet with Defold. Of course I believe someday it will be done but some people or companies should consider time to market too. I would say that Possible and Possible for This Case are totally different. It’s like ideal and reality.

Exactly the plan will improve the usability of Defold. Good news.

And I finally figured out the problem. It was just the building speed is really slow now. I tried to add just an extension to a very small sample project and measured the time. It took around 5 mins. Of course the sample shows the result right away without any extension. It means that the current building time for native is really slow. I waited around 40 minutes and finally Defold made my project. There should be a reason why it’s slow. And I wonder why Defold builds project all again when it restarted. (I guess maybe it’s because the building process is done by a unix/linux session which is connected as a new when Defold is started. Or something like it.)

Take a “vanilla” iOS application without any additional native code as an example:

  • Unity - Generates an XCode project when you build for iOS. You need to have XCode installed and updated. Every iOS build goes through Unity and XCode.
  • Defold - You don’t need XCode installed at all.

If the project contains native code:

  • Unity - The additional native code will be included in the XCode project. I don’t have any experience with Unitys native plugin system to say what kind of possibilities and limitations that exist so I can’t make a very good comparison with our system.
  • Defold - You still do not need XCode installed. You do however need an internet connection so that you can connect to our build server. We will take care of compiling and linking the native code with a vanilla engine and send you a custom engine that will be cached and reused for every subsequent build.

We have tried to make this process completely transparent to Defold users. Users should preferably not have to care about the fact that the project contains native code at all. This is especially appreciated by artists, level designers and others that preferably shouldn’t have to configure a full dev environment to be able to build and test the changes they make to the game.

Of course. I was just pointing out the fact that Defold isn’t a tool that requires the user to always be online to be productive. And even if you have native extensions in your project you only need to be online once to get your custom engine built in the cloud and downloaded and cached.

What kind of libraries did you need and what kind of problems did you run into? And would these problems be more easily solved if you had tried to do the same thing in Unity?

3 Likes

Yes, we cache the result locally between builds.
You either removed that .internal folder (which contains the build cache), or one of the following changed:

  • Defold SDK version (the engine stable version, NOT the editor version)
  • Any of your extensions (i.e. dependencies) was updated
  • Your app manifest was altered

The second point is a good reason why you don’t want to use master.zipas a dependency, but an actual release of a repository.

Ok, good that you figured it out. It has historically mostly been internet related issues (speed, router settings etc).

2 Likes

I’d say it’s the upload speed. (I’m assuming this is the build we’re talking about at ~11:37 in this graph)

The uploaded size doesn’t seem to be more that ~50mb (which is totally fine, we have a max limit of 500mb).
But the upload is taking ~29minutes.

The build time is normal ~30-40 seconds

The response is between 4mb and 13mb (difficult to see in these graphs):

The response write is surprisingly fast too, given the upload time:

2 Likes

What’s the benefit of this? Installing XCode, Android Studio is completely ok for professional engineers and they are even free. All setting them up is less than one day job and Unity and the IDE vendors support the guide. I saw even artists, level designers install them to use Unity or engineers help them. Actually artists, level designers don’t need to install XCode and Android Studio to use Unity, then is it same condition with Defold? They can use Unity right away after downloading it like Defold. Beginner & Amateur vs Professional software engineer. Who does have higher priority for the target market?

I would not say what it is at here because I don’t want to point out the detail problems which can blame the libraries and the authors. Even though it could help them, I don’t want. If I use Unity I don’t even need to make my own plug-in because there are already tons of commercial level things thanks to attending of numerous vendors in the ecosystem. Think about why it is possible in the Unity ecosystem. Marketing? Maybe. But I believe it’s the power of free availability.

The most important reason why I chose Defold is that Defold is relatively light compared to Unity. I believed Defold can reduce a big pain which exists in Unity but currently I’d like to say it was just transforming the type of the pain. So I should keep to find out the crucial benefit of using Defold instead of the other tools. That’s why I told it will be really nice if Defold comes out as a stand-alone completely. But I respect your decision. I was just complaining about the stress what I got in the Defold ecosystem.

Thank you for the information. I understand but I still wonder why today was so slow specially compared to the other days. But that’s ok and now I’m good.

I didn’t know that!

And I appreciate not having to do stuff like this :sweat_smile:

Defold handling this kind of stuff on my behalf is a good thing for me. If it’s costing me something in flexibility or another way - I haven’t noticed or realised yet.

4 Likes

I won’t go into an argument over this. We made a decision to create an editor that doesn’t require any additional tools to get the job done. Unity and other software made other decisions. If that makes them better or worse than Defold I’ll leave for others to decide.

That is a very counter productive position. If there are problems with extensions provided by us or the community it would be of great help to the everyone if we could discuss the problems and hopefully fix them.

Yes, the Unity asset store is a powerful selling point. I won’t argue against that. The Defold asset portal is definitely smaller but I do believe that it contains many of the important to have extensions (available for free).

Could you please elaborate?

2 Likes

No. It’s not about the result of Defold team. As I mentioned I don’t want to say about it. You can not decide easily if it’s counter productive or not. What is productive? All we are human not robot. It’s philosophical thing. This subject is not appropriate for here though. And I know they already know theirs. Everything is not perfect and as you want to keep your way, I’d let them keep their way.

The most significant point is that Defold are handling whole sources of all developers. It means if King.com wants to see what is going on all over the world then they can see. I bet it’s a very sensitive issue to some vendors. I wonder why King.com wants to keep the power of emperor. I was surprised when I heard of the news that Defold is released for free but soon I was surprised again at Defold is still handling all sources of all vendors. On the other hand, Unity doesn’t care the vendor’s source. Unity just gives themselves as it is so that developers can solve the problems while they are using it. It’s natural for most professional developers & vendors in my opinion. That’s why the big vendors give their IDEs purely like Xcode, Android, Unity, and etc.

We’re doing what? No no, you got this all wrong. We shut down or own project hosting service earlier this year. We DO NOT and I repeat DO NOT have access to the source code of your project.

If you are developing a native extension the source code of the extension, not the entire source code of your project, gets sent to the build server. If you don’t want this it is recommended that your pre build your extension code and add the compiled library to your project instead.

3 Likes

In my opinion “I never do it” and “It’s not possible to do it for me” are totally different matter though, that’s good if it works like it. Thank you for letting me know about it. And I already do like it for the extension.

A month ago I tried to include the feature of signing-in for Google Play in my extension and I gave it up because I should struggle to find appropriate jar files in the maven repository. It would be easier & possible if I could use .aar files with Defold. I’m telling about this kinds of experiences. And so now I’m implementing my own leaderboard with nodejs server even though I believe someday you guys will make the extension. Anyway that took a lot of time in my life.

We do have an extension for Google Play Game Services: https://github.com/defold/extension-gpgs

The extension supports sign-in and cloud save. We have deliberately not added support for additional features until they are requested by the community or we get a PR with the new functionality.

4 Likes

I’m a former unity developer, and I permanently moved to defold several months ago and I want to share my reasons with you:

  • To me unity is a one know it all guy that tries to be everything but it ends up being nothing, I mean the amount of complexity that unity added from unity 2015 to this day is incredible, the first thing that caught my eye from defold was its simplicity. and there is such a big difference between published project sizes especially in my experience with android builds.
  • Unity was not 100% free and although you could build entire project with the free version and handling license between version was such a pain at least for me, but Defold being 100% free in any way or form (even plugins) was nice.
  • Friendly Community, was the final bullet that killed unity for me, something that I never find for almost 5 years of unity development, and I say this honestly that with defold community I feel like I am the part of the team and I have such an excitement for waiting and getting new versions, something that I never experienced with unity.

I don’t say defold is perfect in any way or form, for example, I think defold can do more work in terms of documentation and tutorials. especially introducing frequently needed solutions. but the general comparison IMO is not completely fair, I don’t think defold development team is as near big as the unity team and unity has the full support of big companies like Microsoft, Intel, Sony, Nintendo, etc.

8 Likes

I agree and I’d like to say simply Defold is much better than Unity. That’s why I’m using it and struggling here. Also the reason why I shared my opinion is to help(hopefully) the improvement of Defold ecosystem. As I mentioned earlier I see a huge potential of Defold and I don’t deny the fact. I seek simplism for my product as much as I can do. I like Defold because I could discover it seeks simplism in my view.

3 Likes