Colyseus - Multiplayer Game Client & Server

Hi everyone!

If you’re interested in developing multiplayer games, you may find this server and plugin useful. The plugin was tested on HTML5 and OSX platforms, but it should work just fine on other platforms as well.

Check it out:

About the project:
Colyseus is an Open-Source Multiplayer Game Server for NodeJS. It has a server state sync mechanism designed to be simple to use.

More links:

A big thanks go to @britzl for helping out with the WebSocket library and @pkeod for fixing the LuaSec library targetting HTML5 platform.

Cheers!

21 Likes

Looks really good, great job!

1 Like

I’m experimenting with Defold and Colyseus - very cool stuff! One real n00b question: Is there any easy way to implement a seamless/transparent auto-reconnect? When the device goes to sleep, at least on iOS/iPhone, it appears to disconnect the WebSocket connection. Upon wake, I could (I guess) detect that the connection has been closed and reconnect, but that would entail a fresh connection, transferring state “from scratch” (vs. the delta from when it disconnected), etc. Naturally, avoiding doing this would be preferred…

I’m guessing that since Colyseus cleans up connections when the socket is disconnected, this isn’t simply a client-side change, but would require the server to “keep state” for sessions for some amount of time to support reconnections?

For now, I’ll just do the naive reconnection, but as I intend to have a lot of “private server-side state” loaded for each player, I’ll need to ensure that’s persisted outside of the Colyseus session for each player so that when players reconnect, it’s “cheap” to get that private state back and reassociated with a connection.

Just thinking out loud … in case anyone else has run into this and has solved it in an elegant way they’d like to share :wink:

3 Likes

Hi @dossy, thanks for your feedback and the issue you reported on the repo. Glad you’re enjoying using it so far.

That’s indeed a missing feature on the server-side. I’ve created an issue to track this. Some other people also requested the same thing. Hope you can work around this limitation in the meantime.

Cheers!

5 Likes

Using Defold + Colyseus on IPv6-only networks

So, testing my Defold app with Colyseus on my home (IPv4-only) network, connected from iPhone 8 via Wi-Fi to Colyseus running in a Docker container out on Amazon EC2, everything works great.

I decided to disconnect the Wi-Fi and test over my cellular connection, T-Mobile LTE, in the US, which at some point went IPv6-only? Or maybe it’s just Apple/iOS going IPv6-only? Strangely, my older iPhone 5 on AT&T could still reach IPv4 just fine, so maybe AT&T had set up NAT64/464XLAT correctly, but T-Mobile hasn’t?

Either way, trying to connect to Colyseus using colyseus-defold fails, and since I have to turn Wi-Fi off in order to test, I can’t use the Defold Editor to attach via debugger to see what is going on… :frowning: Right now I’m trying to find a SSH client for iOS that can do port forwarding so I can push the ports Defold is trying to use so I can remote-attach to it from Defold Editor over LTE…

Has anyone run into this issue? Is there a solution anywhere? Supposedly, in June 2016 Defold 1.2.83 added support for IPv6, but is there something us game devs have to do to make it work?

I’m investigating this together with @andreas.strangequest. I’m actually not sure what’s wrong yet. The version of Luasocket that we use has IPv6 support as well. I’m not really up to date on the IPv6 stuff so it’s a bit of a learning curve for me and plenty of deep diving into the socket code as well.

I’ve figured out how to use ios-deploy to launch the Defold app on my device and connect LLDB to it over USB, so can at least see the Defold console output… so I’ll be able to dig into this problem, too.

Right now, I’m staring at defold-websocket/client_async.lua self.sock_connect() and its call to settimeout(), because of this documentation:

Also, function that accept host names and perform automatic name resolution might be blocked by the resolver for longer than the specified timeout value.

And, this related issue

I mean, heck, we’re not even checking the return value of sock:connect() - probably because we’re expecting it to function async/non-blocking (thus the settimeout(0)) … but I wonder if that’s unintentionally also preventing the resolver from returning both IPv4 and IPv6 addresses, and causing the resulting connect() to just fail?

Anyway, I’m poking around and now I can get the console log so I’ll be able to see what’s going on. Will check back in when I have something to report…

The connect is asynchronous and the socket is polled for a successful connection like this:

My current theory is that what we’re seeing is this iOS issue:

I’m doing some tests this evening in the hope to confirm this.

1 Like

Ah, yeah, my bad, that’s the standard pattern for async connect (initiate connect, poll with select() for success or failure, yield’ing along the way). Oops.

So, I temporarily commented out the settimeout(0) call, and changed that area to the following:

        self.sock = socket.tcp()
        -- self.sock:settimeout(0)
        local status, err = self.sock:connect(host,port)
        print("sock:connect", status, err)

and the output was:

DEBUG:SCRIPT: sock:connect	nil	host or service not provided, or not known

I don’t know what version of luasocket is bundled in defold-luasocket but this commit is pretty telling:

Although, I commented out the zero-timeout, so in theory it should try all the addresses, but maybe that was implemented in a commit after the version that’s bundled in defold-luasocket? The latest luasocket docs and source (c.f. src/inet.c) implement a socket.dns module, but I can’t seem to find that in defold-luasocket.

As a quick test, I tried hard-coding my IPv6 address in as host to see if this would successfully connect if I fed it the IPv6 address and didn’t depend on DNS/resolver order… and I managed to get it to work if I changed socket.tcp() to socket.tcp6() … but I’m obviously concerned that won’t work if I’m on an IPv4-only network, because I seem to recall seeing that the Lua socket.tcp6() was implemented to bind IPv6-only mode… otherwise we could just use socket.tcp6() everywhere and let it gracefully degrade to IPv4 when necessary…

1 Like

Just another data point:

So, using socket.tcp6() and the FQDN worked on my T-Mobile LTE connection, as well as the hard-coded IP (after I modified websocket.tools.parse_url() to understand proto://[ipv6]:port/ URLs).

Turning my Wi-Fi back on – my home network is IPv4-only – fails as expected because Lua’s socket.tcp6() sets family to AF_INET6 (at least per upstream LuaSocket’s src/tcp.c).

Although socket.tcp() is supposed to use AF_UNSPEC … that’s what the upstream LuaSocket does, and since Defold is closed source, it’s hard to know what version of LuaSocket they embedded and whether it has the latest improvements from the upstream… socket._VERSION from Defold says LuaSocket 3.0-rc1 but that version string was set back on Jun 11, 2013, but the “New agnostic IPv4 IPv6 functions.” commit was on Aug 22, 2015 …

What chance would there be to move the partial LuaSocket implementation completely out of Defold core and into the defold-luasocket extension, including the native code? Would that result in symbol collisions because of the builtin one?

luasocket is bundled into Defold. It’s version https://github.com/diegonehab/luasocket/releases/tag/v3.0-rc1. What defold-luasocket does is to add the missing .lua files (socket.lua is in builtins, but the rest are missing) and also provides mime.core as a native extension. More info here: https://github.com/britzl/defold-luasocket

v3.0-rc1
tagged on Jun 14, 2013 · 110 commits to master since this tag

So, yeah, all the many improvements to upstream LuaSocket haven’t made it to Defold. It would be nice if Defold removed all the socket stuff from builtin and just let the community maintain defold-luasocket and keep it up-to-date with upstream.

2 Likes

This is likely to happen in the long-run but for now we need to live with the current system. I haven’t had a chance to test this properly, but I believe it might be possible to use socket.dns.getaddrinfo() to get a list of IPs to try and connect to:

self.sock_connect = function(self, host, port)
	assert(corunning(), "You must call the connect function from a coroutine")
	local addrinfo = socket.dns.getaddrinfo(host)
	for _,info in pairs(addrinfo) do
		if info.family == "inet6" then
			self.sock = socket.tcp6()
		else
			self.sock = socket.tcp()
		end
		self.sock:settimeout(0)
		self.sock:connect(host,port)

		local sendt = { self.sock }
		-- start polling for successful connection or error
		while true do
			local receive_ready, send_ready, err = socket.select(nil, sendt, 0)
			if err == "timeout" then
				coroutine.yield()
			elseif err then
				break
			elseif #send_ready == 1 then
				return true
			end
		end
	end
	self.sock = nil
	return nil, "Unable to connect"
end
1 Like

I’ve verified this against an IPv6 only network at work and the code can connect to it properly. I’ve released a new version of defold-websocket that includes this change.

7 Likes

I would like to share this here too. Because It is possible that other dependencies(like defold-websocket) or Defold’s cache may cause this problem:

I am just using default example. When server is not available, onError message “mostly” doesn’t triggered. Sometimes it works when I build and then rebuild the project, but I couldn’t find any clue or pattern about this. Its possible that other dependencies may cause this problem or maybe built in cache of Defold.

This is the client output when there isn’t any server and onError doesn’t triggered :

{
  roomStates = {
  }
  roomsAvailableRequests = {
  }
  requestId = 0,
  on = function: 0x0a1b70e0,
  connection = {
    emit = function: 0x0aaf9d60,
    _enqueuedCalls = {
    }
    on = function: 0x0aaf9d80,
    is_html5 = false,
    off = function: 0x0aaf9f40,
    _on = {
      close = {
        1 = function: 0x0aafa1f0,
      }
      error = {
        1 = function: 0x0aafa140,
      }
      message = {
        1 = function: 0x0aafa180,
      }
      open = {
        1 = function: 0x0aafa0a0,
      }
    }
    state = CLOSED,
    listeners = function: 0x0aaf9d40,
    endpoint = ws://localhost:8080/?colyseusid=xmjmMr3lo,
  }
  rooms = {
  }
  hostname = ws://localhost:8080/,
  connectingRooms = {
  }
  _on = {
  }
  emit = function: 0x0a1b7210,
  listeners = function: 0x0a1b71f0,
  off = function: 0x0a1b7100,
  id = xmjmMr3lo,
}

There isn’t any server and onError triggered :

{
  roomStates = {
  }
  roomsAvailableRequests = {
  }
  requestId = 0,
  on = function: 0x12f4f0e0,
  connection = {
    ws = {
      step = function: 0x13892d10,
      sock_close = function: 0x13892860,
      on_close = function: 0x13892880,
      send = function: 0x13892c30,
      on_disconnected = function: 0x13892d70,
      sock = tcp{client}: 0x13893630,
      sock_send = function: 0x138928a0,
      close = function: 0x13892cc0,
      on_connected = function: 0x13892d50,
      on_message = function: 0x13892d30,
      connect = function: 0x138929f0,
      state = CLOSED,
      sock_connect = function: 0x13892830,
      sock_receive = function: 0x138928c0,
      receive = function: 0x13892c80,
    }
    emit = function: 0x13891d60,
    _enqueuedCalls = {
    }
    on = function: 0x13891d80,
    is_html5 = false,
    off = function: 0x13891f40,
    _on = {
      close = {
        1 = function: 0x138921f0,
      }
      error = {
        1 = function: 0x13892140,
      }
      message = {
        1 = function: 0x13892180,
      }
      open = {
        1 = function: 0x138920a0,
      }
    }
    state = CONNECTING,
    listeners = function: 0x13891d40,
    endpoint = ws://localhost:8080/?colyseusid=xmjmMr3lo,
  }
  rooms = {
  }
  hostname = ws://localhost:8080/,
  connectingRooms = {
  }
  _on = {
  }
  emit = function: 0x12f4f210,
  listeners = function: 0x12f4f1f0,
  off = function: 0x12f4f100,
  id = xmjmMr3lo,
}

I just test the defold-websocket and I can confirm that it works as expected.

ws:on_connected() returns an error with ‘closed’.

Also ws:on_connected returns ‘closed’ on colyseus (connection.lua)

self.ws:on_connected(function(ok, err)
    self.state = self.ws.state
    
    if err then
      pprint(err) -- <- returns closed
      self:emit('error', err)
      self:close()

Ok. Does this mean that self:emit(“error”, err) doesn’t work or?

Yes, looks like emit is not working as expected. Sometimes it work sometimes not.

Ok, makes sense than. Have you opened a ticket in the Colyseus repo?