Binary serialization/deserialization

Questions
1

I’m new to Lua and I’d love some advice. I have a server on Go, and Defold communicates with it through websockets using britzl’s library. Serialization is easy in Go, but I’m having troubles on the Defold side of it.

Right now, let’s say I need to deserialize an int32 in Defold. This is what I do:

function ReadIntFromStream(message)
	local result = ""
	for i = 1, 4 do
		result = result .. message:byte(i)
	end
	return tonumber(result)
end

After that, I cut off first 4 bytes from the “message”, before passing it along to other methods.

message = string.sub(message, 5)

All of this looks hacky to me. Is this the correct approach at all?

If I need to deserialize structs, do I have to manually do it for each element of the struct? Perhaps writing a C++ extension would be the better approach, especially for structs, since this is where I can also declare them? (at least from what I understood from reading about C++ extensions)

2

There’s a msgpack implementation for Lua that you could use on both ends:

1 Like
3

Msgpack is a self-describing format, which means slow performance. But thanks, it set me off on the right path and I found ProtoBuf and FlatBuffer. I’m going to try both, but they look like exactly what I need.

3 Likes
4

On the subject of msgpack, it looks like this implementation is more interesting: https://fperrad.frama.io/lua-MessagePack/
(sources at: https://framagit.org/fperrad/lua-MessagePack )

It relies on LuaJIT when available (and it’s the faster version), but when it’s not available, it falls back on pure Lua 5.1

2 Likes