Extension-websocket

[Mathias_Westerdahl]

We’ve created a new extension extension-websocket, that will supercede our old extension (defold-websocket).

The extension is ready for testing now, and we’d like to invite you to test if you use websockets in your projects.

This extension has some advantages over our previous extension.

• Build size (~30% smaller)
• Uses native code (C/C++)
  • We can control the socket settings more easily
• Possibility for threading (in case we need it)

Here is a comparison between the old and new websocket extensions, as well as the vanilla engine. (Numbers in bytes)

	defold-websocket	websocket-extension	vanilla engine
OSX	6 736 020	4 070 312	3 963 408
iOS	3 937 564	2 682 406	2 644 899
Android	5 781 607	3 319 411	3 212 694
Windows	7 216 128	4 855 808	6 170 624
Linux	6 291 120	3 891 208	3 871 648

Currently, the only known issue is that the secure connections doesn’t work on HTML5 currently, but we’re working on that.

We’d like to know any issue you find, and also what you think of its performance compared to the old extension.

Please report any found issues in the extension repository.

[britzl]

@endel we would really value your opinion on the new extension seeing as you rely on defold-websocket for Colyseus. The new extension will give a performance boost and also reduce the number of dependencies required.

And @BunBunBun we’d appreciate your feedback as well since you use WebSockets in Raft Wars right?

[BunBunBun]

@britzl very interesting! thanks for inviting here. In Raft Wars we use Colyseus also, we will check out with @endel tomorrow!

btw, does I understand correctly, that this extension need to use instead 2 libs?

[britzl]

btw, does I understand correctly, that this extension need to use instead 2 libs?

No no, the extension is completely self-contained. It uses the socket code from the engine, exposed as dmSocket and dmSSLSocket to extensions through the Defold SDK. The actual WebSocket layer with handshake code etc is based on wslay.

[novabyte]

Nice work @Mathias_Westerdahl. Do you know if the TCP socket in the native code is configured with TCP No-Delay or Quick ACK on?

[endel]

That’s great news @britzl @Mathias_Westerdahl! Thanks for letting us know this early!

I’m trying to use the new extension, but I’m getting this error:

• I did set up the https://build-stage.defold.com/ (Preferences -> Extensions -> Build Server)
• I’m on Mac OS 10.15.5
• Just upgraded Defold Editor to 1.2.172

Should I set-up something else to be able to use the new build system? Cheers!

[Mathias_Westerdahl]

Thanks!
Currently it’s using TCP_NODELAY, but I’m also in the process of adding a similar function for TCP_QUICKACK, it should be available in the defold beta later today. However, I’m not familiar with the quick ack functionality, so I’m not sure if one needs to set it once, or several times. I hope someone will help with this info
Also, TCP_QUICKACK isn’t supported on iOS/macOS and Windows.

[Mathias_Westerdahl]

Ah, I should have been more clear. You need the 1.2.173 beta of Defold too to use it, since we’ve added a lot more functionality to the dmSDK (e.g. socket.h)

EDIT: Defold 1.2.173 BETA

[britzl]

Just upgraded Defold Editor to 1.2.172

Use the latest beta editor when testing this! Download from http://d.defold.com/beta/

[britzl]

However, I’m not familiar with the quick ack functionality, so I’m not sure if one needs to set it once, or several times. I hope someone will help with this info

PR for quick ACK is here: https://github.com/defold/defold/pull/5162

I’m also not sure of the implications of quick ACK. is it always beneficial to have it enabled regardless of how the socket is used (I’m thinking HTTP connection vs socket used in a game)? I also saw in a SO post that the quick ACK socket option can be changed by the system and that we might have to set it after every send.

[endel]

Thanks for the clarification, it works using the beta.

So far the Desktop build is working just fine with Colyseus, but the HTML5 build is not connecting to the right endpoint (notice the URL segments and query string in the console log)

I’m going to provide some feedback on this thread as I face the issues… Hope not to annoy you all!

[britzl]

the HTML5 build is not connecting to the right endpoint

The HTML5 version has a different code-path which explains the difference. We’ll look into this on Monday.

[Mathias_Westerdahl]

@endel I’ve uploaded a new version with a separate code path for the html5 code. I did make a quick stab in the dark by adding the path to the url when getting the address (trying to compare the functionality with the old extension), but I currently have no way of knowing if that works. You don’t happen to have an example echo test server on such an endpoint I could test with?

EDIT: To use this, update the beta editor (or bob.jar) and refetch the extension.

[endel]

Awesome, thanks @Mathias_Westerdahl! Now it works fine, I’ve added a note to your commit here: https://github.com/defold/extension-websocket/commit/071adac853b2844e2fb5eb0ff8a20699c17f015a#r42065637

Going to do some more testing during the week

[Mathias_Westerdahl]

Thanks for testing!
I’m glad it worked, it was a bit of a stab in the dark

I’ve uploaded a new version of the extension without the superflous ‘/’.

[GreenArrow]

Tried this in my colyseus project (using @endel’s new websocket-extension branch). It seems to work without problems in the html5 build. But for some reason, I am getting an error when running from the editor (Project/Build). I am using Windows 10, Defold 1.2.173 beta and latest websocket extension.

ERROR:WEBSOCKET: Failed to find the Upgrade keyword in the response headers
ERROR:WEBSOCKET: Response:
"HTTP/1.1 101 Switching Protocols

Server"

DEBUG:SCRIPT: websocket error:	Failed verifying handshake headers:
Failed verifying handshake headers:
rver

Thoughts on what could be the issue? I am connecting via wss, nginx has been configured for http upgrade. It works fine with my html5 build, but not when running from the editor. I don’t have this issue with the old defold-websocket setup.

[Mathias_Westerdahl]

The Html5/Emscripten sockets are actually already websockets, so they don’t have to do the handshake.

For the other platforms, the extension expected the websocket handshake return headers as specified here:
https://tools.ietf.org/html/rfc6455 (see page 7)

I’ve already seen one discrepancy from that standard, and looking at your printout of the headers, you server doesn’t really conform 100% either:

"HTTP/1.1 101 Switching Protocols

Server"

Did the old extension (defold-websocket) work for you?
Our goal is to make them as close as possible in supporting the servers they already work with.

Perhaps @endel has some insight into this?

[GreenArrow]

Did the old extension (defold-websocket) work for you?

Yes. The old extension works for me and I don’t have this same issue.

[Mathias_Westerdahl]

Using this tool, does it successfully connect to your endpoint?
https://www.wss-websocket.net/

I’m wondering if I should skip checking for the “Connected”/“Upgrade” headers, even though they’re part of the standard.

[Mathias_Westerdahl]

Also note that in his post, the returned headers are following the standard:

Extension-websocket (beta testing)

Thanks for the clarification, it works using the beta. So far the Desktop build is working just fine with Colyseus, but the HTML5 build is not connecting to the right endpoint (notice the URL segments and query string in the console log) [43] I’m going to provide some feedback on this thread as I face the issues… Hope not to annoy you all!

So the question is, why isn’t your request returning those headers?