OK then, but how it works? We need to backup .internal/cache/engine-archives after created a successful build?
Yes
Sure, we will try it out.
Yes, I am using VPN to access google & Defold
I have no any proxy or VPN
I don’t think this issue is related to network access, because I can create macOS build in same environment.
Well, the weird thing is that it times out when it’s supposed to be writing the result back to the user. The resulting file is ~10mb, and it times out due to being idle for 30 seconds. This suggests that something about the communication is going awry, possibly because the entire upload/build takes so long. It’s good to know that you are building the osx version on the same setup with full success.
2 Likes
We’ve just now rolled out a new server, with out next try to increase the idle timeout. Not entirely sure if that will help, since it really shouldn’t timeout.
At the same time, we’ve added some more metrics on the server, and what we can see is that the upload of the files is taking ~5 minutes. It’s trying to upload ~60mb of data, and when it’s done building, it should download the resulting zip file which is less than 5mb.
1 Like
After adding more and more native extentions in the project, the building for windows is always failed today.
It seems to be the failure is in the download phase. You may change the timeout setting to make sure that the client waits a little bit longer, although the server side spends more time for windows.
Log FYI
Blockquote
2018-06-21 20:19:54.930 2007131 [JavaFX Application Thread] WARN editor.app-view - {:line 499}
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570)
at editor.engine.native_extensions$build_engine_archive.invokeStatic(native_extensions.clj:196)
at editor.engine.native_extensions$find_or_build_engine_archive.invokeStatic(native_extensions.clj:208)
at editor.engine.native_extensions$get_engine.invokeStatic(native_extensions.clj:266)
at editor.engine$get_engine.invokeStatic(engine.clj:171)
at editor.engine$launch_BANG_.invokeStatic(engine.clj:178)
at editor.app_view$launch_engine_BANG_.invokeStatic(app_view.clj:413)
at editor.app_view$launch_built_project_BANG_.invokeStatic(app_view.clj:460)
at editor.app_view$fn__48356$fnk48348_positional__48357$fn__48358.invoke(app_view.clj:550)
at editor.app_view$async_build_BANG_$fn__48338$fn__48341.invoke(app_view.clj:523)
at clojure.lang.AFn.run(AFn.java:22)
at com.sun.javafx.application.PlatformImpl.lambda$null$173(PlatformImpl.java:295)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.application.PlatformImpl.lambda$runLater$174(PlatformImpl.java:294)
at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at com.sun.glass.ui.win.WinApplication.lambda$null$148(WinApplication.java:191)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
… 21 common frames omitted
Yes, we just increased it. Please try it again.
Note that it times out on idleness, so if the receiving end doesn’t receive bytes anymore, I’m not sure what to do.
2 Likes
The building is still failed.
The client(editor) may close the connection due to the read timeout.
You may try following suggestions:
- Check the client(editor) code if there is a socket.setSoTimeout(int timeout) setting which might cause a disconection from the client.
- Add a keep alive in the server socket to make sure there is no longtime silent connection.
For now,The timeout frequency is lower, but it is still too slow for building.Especially debugging the native extension.
I suggest deploying our own local build server,That will extremely boost the development efficiency.
We are working as fast as we can to solve the issue.
As for releasing our source code for creating a local build is quite a huge undertaking, it will not happen anytime soon (if at all). Instead we will focus on making the main client-server functionality work robustly instead.
3 Likes
I mean not releasing your source code,Just the running binary and some setting files.
And the sdk, all part of the code which we’re not ready to release yet.
2 Likes
I encounter 3 build time out Just Now, The issue seems not to become better. Here is my suggestions:
1.For win32, use dmengine.exe + dll to extend the engine.native extension in dll which is built locally
2.Caching the .lib for win32
3.In the extensions perference setting, add a timeout setting,Default to 10 minutes, then I can set it to 20 minutes,even 1 hour.
I’m still timeout all the morning.Any good news?
@Mathias_Westerdahl, can you release a defold editor with verbose logs to us, to help debugging with this issue?
The earlier attempts has apparently not improved things by much. Here’s what’s on the horizon next
- Try to increase the Editor connection timeout.
- Complete the faster Win32 builds on the server (not very much much left) (DEF-3191)
- Start implementing the optimisation of caching the uploaded (DEF-2410)
The caching task is a bit tricky, since our backend person is currently away on vacation for another week.
Q: Comparing to your MacOSX builds: How many megabytes is the MacOSX build? (i.e. the libraries that you upload). And how many megabytes is the Win32 libraries?
As for a debug build of the editor, you should be able to see the debug logs by showing them directly in the editor:
2 Likes