Why on earth does sys.deserialize throw errors?

Questions
1

it really should return nil on a parser failure, like most functions that handle potentially unreliable data.

2

pcall is your best friend:

3

pcall is bad for performance and a cop out for improper exception handling, and does not answer the question.

did nobody think that you’d try to parse corrupted data?

4

In Lua, pcall (protected call) is the primary mechanism for handling errors and exceptions.

sys.deserialize is a function implemented in C++, so pcall doesn’t affect the performance at all.

5

pcall is the primary mechanism for handling errors

no it isn’t. handling invalid outputs is. errors in lua are only thrown due to severe programmer negligence. when io.open tries to open a file for reading which does not exist, does it throw an error? of course not, that’d be silly. it returns nil and it lets the programmer deal with it how they see fit. only if you try to read from the non-existent file will an error will be thrown. same with indexing a value in a table which hasn’t been set. you get nil, and must adapt.
pcall is by far the least useful error handling mechanism in lua, and only serves to prevent an error from aborting execution.

pcall doesn’t affect the performance at all.

yes it does. function call overhead is real, C or Lua function.

6

Have you benchmarked sys.deserialize with/without pcall?

2 Likes
8

Have you benchmarked

no, and i don’t need to. there is no version of lua where a(b()) is faster than b(). function calls are the single most expensive operation in lua.

Both methods see widespread adoption

i get that, and there’s many cases where throwing an error is 100% understandable, but i don’t think that this is one of those cases.

Performance complaints [are] a non-issue.

i disagree. every clock cycle counts, especilly when they’re so easy to get.

i’m using serialize for sending data via UDP. speed is my top priority.

9

If you ran a benchmark, you would see that the difference is in nanoseconds.

if performance matters to you, then in my opinion the obvious choice is using your own data format via Protobuf (there are extensions) or similar options.

io.open is based on the design of POSIX functions, which is why it behaves that way.

By the way, string.unpack (from newer versions of Lua, not in Defold) also raises Lua errors, and it’s a built-in Lua function used for data deserialization as well. So your arguments are fundamentally incorrect.

2 Likes
10

Nobody maybe acknowledged your issue related different API expectation, that is an issue for you, so sorry for that, people here tried to give solutions first, but if you feel like this should be improved, please raise an issue on Github, so then it will be discussed by the Defold team.

There’s also nothing stopping you against changing this code in the engine code for how the sys.deserialize works and build version for your game, if this is really critical for you. If you decide to, check out readme on Github (or add an issue there as suggested above) and don’t hesitate to write if there will be any problems with this:

Also, maybe moving just the code related to critical UDP handling to a Native Extension might be a better and faster option then? I imagine there might be more critical sections than only deserialization probably.

Hope this will help a bit :wink:

2 Likes
11

Also raised as an issue on GitHub: sys.deserialize throws errors on failiure · Issue #11499 · defold/defold · GitHub

12

I did a measurement:

DEBUG:SCRIPT: No pcall took 0.035672903060913 for 100000 iterations
DEBUG:SCRIPT: With pcall took 0.035229921340942 for 100000 iterations
DEBUG:SCRIPT: With pcall and error took 0.065108060836792 for 100000 iterations

100000 iterations on a Macbook Pro (Apple M3 Pro).

Calling sys.deserialize() with and without a pcall in the case of no error takes the same amount of time. If an error is thrown it is twice as slow.

7 Likes