Building a Multiplayer Server

[dlannan]

Building a game for for multiplayer online use can be a challenge. A couple of months ago I started using Nakama, and it is a brilliant system for mp gaming and chats, etc. Its very well thought out and provides a very large array of features.

I want to be extremely clear about my use-case, Nakama wasnt really the right “fit” for the game I was making, and I really only found out that while building the game and using Nakama. The type of game I am building is a group discussion game where the webapp allows people to control the game flow and outcomes. In this manner I wanted a system that was extremely simple to join and create a game without all the login/handshaking/user management that alot of server systems have - Nakama has this. These systems are very important for things like realtime fps or high frequency controlled data sharing - but it isnt really suitable for the game I was making.

To explain, what I was looking for were the key following features:

• No login or signup process - the app handles the device login, and handshakes for the auth.
• Can easily generate a game from a name - the internal systems allow direct name lookup for the game.
• Players can drop/in out of game lobby easily (backend handles all of it)
• Modular game executions on backend (Ideally a luajit vm running it)
• Admin panels and control to monitor and modify modules, games, and user profiles.
• Wanted it to be properly open source(MIT)
• A nice Defold client API.
  Its not an exhaustive list, but some of these things proved difficult to do in Nakama without spending alot of time building an RPC system to create the results I needed.

Additionally I wanted a simple SQLite based backend DB that allows me to snapshot state of the server if needed. I had built a number of luajit + Sqlite systems before so it seemed like a reasonable choice for the system type.

SWAMPY was born. Dont ask about the name, it was a tag I used and now its stuck
The aim of SWAMPY is to meet the above features using a luajit based environment entirely written in lua. Yes, this is going to get a bit weird from here on in. The entire server is written in lua using a great toolkit and framework I have used in the past called luvit (www.luvit.io). It should run on pretty much any platform - technically you could run a server on your phone, which would be a bit odd.

The only real dependency is the libsqlite library used. I may compile that into the exe at some stage if I get the urge to. The whole thing, website + admin + luvit + luascripts = 17MB. Yes. 17MB. And this is a debug build, so expect the release to be < 8MB. This is another reason why I am a big fan of luajit based systems - cross platform, extremely tiny, and extremely fast with low resource overheads ( Mem use < 48MB with debug ).

SWAMPY is in its early stages of alpha testing. Most of the features are good to use and I am building my game to use them and trial it. Once I have deemed it stable enough to use, I’ll put it up on github for people to do whatever they want with it (along with the Defold Client).

SWAMPY still needs many things done to be properly user friendly too (especially on the admin side):

• A decent UI theme. I need to spend some time on this and make it look nice.
• Remove the placeholder data and put the proper data in (I have it, but not a priority atm).
• Add a couple of nice admin login features - Google, Insta and FB auth as well 2FA would be good.
• Data editing for the Sqlite db - this is high on my list of todos. It will probably be done sooner than later.
• Many module and user related features…

I made this dev diary so that I can update here, and give some general idea about what I have been building to be used in this forum article at a later date: Peer 2 Peer Networking and State Buffers
(No I hadnt forgotten - life has gotten in the way).

My aim is to have this and the forum post all sorted over the next month (eta end of August) all things going well. I hope to have SWAMPY ready for consumption in the next two weeks along with my game Soul Survivor … that may be a little optimistic, but it should be doable.

For the time being, here are some screenshots of SWAMPY in the web admin, and a couple of screenshots from Soul Survivor. Thanks for reading this far!
SWAMPY Dash - ignore the stats, they are just placeholders

SWAMPY Profiles - users connected that are joining a module (game type).

SWAMPY Games - list of running games

Screenshots of Soul Survivor (wip)

Oh. Another interesting note. Soul Survivor is entirely written in Defold + imgui. There are no Defold ui elements used in this system. And its quite ui intensive - the game elements. I might write a whole dev diary on Soul Survivor and imgui - thats been a pretty interesting process, and its alot more powerful than I thought it would be.

[SpencerWF]

Sounds fascinating. I’m assuming this game is designed for an app on the phone? I think it’s a cool idea, and has given me an idea for another game. Thanks for posting this!

What inspired you to this game? How do players start a game/invite others? What other kinds of games do you think would work for SWAMPY?

[dlannan]

Hi Spencer. The game is designed for a phone and web. I’ll be adding voice chat so people can play remotely too (thats further down the track though).

Its kinda inspired by a whole tun of board/card games me and my family play. Like Secret Hitler, the Werewolf one (forget the name), a game called Chameleon which can be harsh … and some others. The goal was to make something that is fun in the creation stage, and then also in the discussion stage. Most discussion games have problems with arbitration, so we added a judge… and it works well.

We are making a prototype card game and will hopefully have a kickstarter in a month or so.

Starting a game is built a bit like some other phone games. The app finds a server (atm the only 1, but it will use geoip and find closest). A player then simply “joins” a game with a nominated name - I generate random ones if people want. Then other players join by simply providing the same name. At the moment theres no passwords so you may land in someone elses game if its in lobby mode … thats easily fixed if you want a private game (passwords). Once the minimum amount have joined the original creator can start the game, and others can click “ready” to allow the creator to start it.
Some of the gameplay also occurs in the lobby… so while you are waiting you can not get too bored while waiting

SWAMPY really was made for this type of game. However, its all REST / API based at the moment, so it would work well for simple event styled dropin games. Like board games and card games. It can do more complex ones, because you can write sockets in the modules and have clients join to them, so its not impossible to write a rts or fps in it if you really want, but I’d need to make sure the API has the features needed for it.

[britzl]

The server side solution sounds like something @88.josh could use for his digital version of Malas Decisiones. @88.josh: I can’t remember if you ever completed the digital version or not?

[dlannan]

Update: Have been running the server for two weeks now for dev testing and it seems quite stable. Going to put together a little sample for people to test with. Due to house issues, this has all taken far longer than Id like, but at least its working and the game soul survivor is looking fine too so I hope to wrap this all up in the coming couple of weeks.

[dlannan]

Ok. The server is up. It is currently empty (I removed my IP from it). I will be adding a little test MP game using Defolds demos. Probably a little twist on the Tank demo

GitHub - dlannan/swampy: A luvit built multi player server with a web admin...

If you have any questions raise an issue or msg me. It should be fairly simple to get working. Highly recommend using Docker-Compose or, if you want to dev you can run it directly. I’ll add some more docs tonight in the swampy folder on how to do that.

[britzl]

@dlannan

Ok. The server is up. It is currently empty (I removed my IP from it). I will be adding a little test MP game using Defolds demos. Probably a little twist on the Tank demo

Really cool! Thank you for sharing!

[dlannan]

Ok. Theres now a simple sample module. Its not fully complete, but should be in a day or so.
Once thats done, there will be a little War Battles example added as well - probably a week away.

GitHub - dlannan/swampy-defold: API for the SWAMPY server system using the...

The warbattles is here (not complete).

GitHub - dlannan/defold-warbattlesmp: War Battles MP

This should be wrapped up in around two weeks. Then I’ll sort out the state buffered networking sample using this. Sorry its taken so long, work and life… you know.
Just to let you know our little dog died (of 18yrs), we renovated and sold our old house of 12 yrs, Started a new major project with fmad.io (exciting, but lotsa hours) and many family events in between… I know everyone has things going on, I just want people to know I am committed to Defold and will get these projects sorted as best I can.

And again… really love working with Defold and I am very appreciative of the hard work everyone has done to make such a nice system to work with. Kudos to you all!

[dlannan]

Test example all working nicely. Sample module is able to be used as a basis for games.
Currently event/poll based and a few features yet to be added:

• Module per Luajit env
• Websocket sample for realtime games (FPS, RTS etc)
• Data exchange example in the module

Some of these will be added over the coming weeks (for use with the War Battles MP demo).
Otherwise its in a fairly good condition. Feel free to hassle me if you need anything specific.

[dlannan]

Added a bit more to the admin. Some tidy up and some gfx.
Fixed a number of issues with the example. And adding in a module for War Battles MP.
On track to have War Battles MP working by end of week - fingers crossed.

[dlannan]

Ok. War Battles MP is starting to work.
I need to make sure everyone understands what this is and what it is.
What is isnt:

• A useful RTS/FPS example
• A high speed low latency game server and client
• Hugely scalable network game system for realtime games

What it is:

• An example of http (yes, you read right) only realtime game server
• A starting example of how to do multiplayer state sharing (badly atm)
• A demonstration of the module system for swampy to run a gaming session with a different style of game (non event based)

I’ll add in the tanks (driven from game module) tomorrow and that will probably be it for this example and setup for the time being. People are welcome to use and abuse the code/systems as they need

Here’s a little sample of two players in a game. The lag is roughly a 300 ms round trip with interpolation. And the http req’s stack up a bit. I’ll improve this, it should end up reasonably snappy.
There are also a bucket load of bugs to iron out. Nothing too nasty though. Enjoy.

[dlannan]

Still working on this. Trying to finish off before end of November. I went a bit stupid and built a little pathing system for the tanks. Here’s their paths overlayed on the map:

I might add a little native extension for this. Might be useful for people to make AI paths.

[88.josh]

Hi,

Not sure what professional experience you have but I’m looking for some tutoring to help build a simple server that can use GET and POST to store and retrieve data for an online learning platform.

Let me know if you think you can help me out (paid work!)

Josh

[dlannan]

The tanks are on the move!

[dlannan]

Updated SWAMPY quite heavily to be more Heroku friendly. Has been split into two.
Docker: https://github.com/dlannan/swampy-docker
SWAMPY: https://github.com/dlannan/swampy

Heroku instance is here to play with:
http://ancient-garden-51927.herokuapp.com/
login: admin@test.com pw: admin

Admin is a little broken. It keeps kicking the user out. Which is a little annoying
The module system should work fine though. I’ll update as this game example nears completion.

The Heroku setup should work out of the box, which means you can run your own server very easily.
I’ll add some docs to the repos readme so you can follow those instructions if you want to spin one up.
Should have this wrapped up in a couple of days now.
Hope people have enjoyed the trials and tribulations … I’ll update the other thread too because I have implemented a simple state system which shows how these things can be timed. There is also a bit of predictive examples and time based systems which the tanks use (only uses a start pos and a timer).

[dlannan]

Update: Dont use the above settings. It wont work. A https/wss setup that needs multiple ports is not available on Heroku, so I wasted a tun of time on that (the http is easier, you can route but its insecure).
New info:
https://swampy.kakutai.com:5000/
login: admin@test.com pw: admin

Admin is still a little broken. I will fix this. I wasted a tun of time trying to make a special Heroku version people could use, but it was too much hassle. Ideally, I could split the admin and the websocket system but that then creates other work to be done. For a ‘simple’ system it would expand more time. Easier to just put on a VPS and run it as is

Again, things are thus delayed by a few days. The docker is all working well so feel free to try that out if you want to run your own. Just use:
git clone --recurse-submodules https://github.com/dlannan/swampy-docker.git

Then run:
docker-compose up -d

You will then have an admin server running on port 5000 accessible: https://myserver.com:5000/
And the websockets will be running on 11000->11100 (this is a websocket setup for each possible game limited to 100 games of course).

I’ll put up a client version on itch.io once its all working later this week. Has been fun doing all this. Thanks for reading.

[dlannan]

Uggh. I feel like I keep bumping into stupidities. Isnt coding fun

Anyway, the admin is fixed, but the Websockets on the server side are broken. I need to do some heart surgery on the code so it might be a few weeks now. Which is a little more than annoying. At least when complete swampy should be a very solid little server

[SpencerWF]

Hello @dlannan,

Still loving the idea of a multiplayer interface designed not for high-speed games but more the board game experience.

Can’t wait to try it out when I get a chance.

Cheers,
Spen

[dlannan]

Ok. Good news. Websocket survived the heart surgery (just) and now its happily talking properly with Defold (not dropping packets and failing fragmentation). The tanks are now working as they should - well, within reason, they have broken orientation
I hope to roll over the multiplayer movement and rockets to websocket tonight/tomorrow and it will be done! (or close enough to).
The last 5% always takes 95% of the work…

[dlannan]

Ok. Its alive and kicking. And its fairly complete now. There are a number of little things that I will fix over the coming weeks. But overall its a usable little mp game + server.
Notes:

• There is about a 200ms lag because of the 100ms round trip for some of the data and I fixed certain updates to 10Hz (because this is a fun little test)
• There is some odd issues with the serialization lib Im using - smallfolk. It has some weird probs. I will revisit this sooner than later, because I want to use it elsewhere.
• The admin is still rudimentary. Again, this will get some love soon because Im using it for my card game which I think I will open source soon too.
• The modules can and do crash the server. This is because they are not yet running in their own lua process - again, thats coming for the above reasons.

Heres a little vid. Websockets are certainly much better than http. Id expect with some tweaking (I serialize way too much data) it could be very fast and quite usable in a RTS or even possibly a FPS - but that would need some work on the prediction routines that are unfinished in the module.

Feel free to use it as you wish. It should all be MIT. Thanks for reading, it was a fun little project and I’ll be writing some updates in the other thread about state management.