Pigeon - easy and safe messaging library for Defold

[Pawel]

Hi!

I wanted to share with you a library I’m working with for easy and safe messaging in my Defold projects. At first, I was using Dispatcher by Critique Gaming and Pigeon is its evolution.

GitHub - paweljarosz/pigeon: Pigeon allows easily and safely manage posting...

It was meant to be my birthday gift to the community, I am only late few days, because, you know, documentation! (at least it is extensive and comprehensive, I hope!)

Pigeon is very easy to use, and you can directly replace all your msg.post with it even, but it offers of course much more, main advantages are:

• Easy to use subscription system
• Safe checks for defined messages (it can check if data contains proper keys and type is correct)
• Comes with all Defold system messages defined (so you won’t be able to send a wrong message using it)
• You can define your own messages as well. Defining message data is optional.
• You can send messages to all subscribers or to specific urls directly (while still checking data corectness, if the message is defined)
• You can use strings or hashes. (All strings are pre-hashed internally anyway thanks to amazing Defold-Hashed by @sergey.lerg ).
• It comes with documentation, examples and some functional tests!
• I use the module in Witchcrafter!

Simplest usage:

pigeon.subscribe("test_message") -- subscribe where you want the message to be received
pigeon.send("test_message") -- send to all subscribers

With message definition (data is verified when sending, so receivers can assume they always get correct data and no additional check have to be done in on_message):

pigeon.define("test_message", {test_value = "string"}) -- define message to require table with at least one key "test_value" of type "string"
pigeon.send("test_message", {test_value = "Hello World!"}) -- message data is verified before sending

Replace Defold’s msg.post with pigeon.send_to:

pigeon.send_to(msg.url(), "test_message", {test_value = "Hello World!"})
-- is same as:
msg.post(msg.url(), "test_message", {test_value = "Hello World!"})

Then in on_message you can receive messages as usual.

For 'in-depth-ers", there is letters module that defines letters (messages) to be checked by Pigeon. You can directly extend this module, instead of defining every message with pigeon.define() in runtime

EDIT:
Submitted to Asset Portal

---

Also, I wonder what do you think about setting dependency to log, but explicitly, by user?

I use log everywhere and for the purpose of this library I removed this dependency (but left Defold-Hashed, as it is really small and essential actually). You can replace internal logging system, that by default uses just print() function with log module by @Pkeod with call:

local log = require "log.log"
pigeon.set_dependency_module_log(log)

And it will be printing data with correct level, tag and could also save logs in file, depending on how you set up log

I already have reworked also DefSave and am thinking about using the same idea for logging there. I looked up this idea a little bit from Monarch, where logging is enabled, by actually assigning function print to internal log, otherwise nothing is happening (logs are disabled).
As logging is generally a good part of any library, this could be a unified way of adding dependency for logging.

What are your thoughs? Are you using log or other modules for logging?

[WhiteBoxDev]

Thanks for this!

@Pawel

Safe checks for defined messages (it can check if data contains proper keys and type is correct)

Yay, that’s been a point of annoyance once you reach a high number of custom messages. In a way, this reminds me of moving from JavaScript to TypeScript for large applications.

[Pawel]

Exactly, it’s only a runtime check, but it’s something that simplifies development a lot and when I’m sure some messages are fixed and correct I can easily replace them with barebone, correct msg.post.

Static languages have advantages over dynamically typed ones, but there are advantages of the latter as well

[mako]

This looks great! My main issue with the built-in system was the reliance on hashes and URLs, which can be tedious to work with sometimes.

I can see this being the easy alternative to messages like Monarch is for screen switching with collection proxies.

[Insality]

Thank you for sharing this well-described and useful library!

I tested and read the code of Pigeon and found several points:

• If we use the “hook” callback for instant feedback, we should be careful. Since this hook will be called directly from the place where “send” is triggered, it means if we call pigeon.send in go place and catch it in gui, it will result in a context error:
  You can only access gui.* functions and values from a gui script instance (.gui_script file)
  It can be tricky, but we can check the equality of the context like this:

local function send_event(target, event_name)
	local current_url = msg.url()
	current_url.fragment = nil
	local target_url = msg.url(target)
	target_url.fragment = nil

	if current_url == target_url then
		-- One context, we can call functions directly
	else
		-- Different context, we should use msg.post()
	end
end

At least we can show the warning or handle it in some other way.

• Note that if we set both the “hook” and “url” parameters, only the “hook” parameter will be used.

• Hook callback functions receive two arguments: the message_id:hash and message:table. It might be useful to pass the first argument parameter in the subscribe method. For example:

local function close_function(self)
    print("We got correct self!", self)
end

function init(self)
     pigeon.subscribe("test", close_function, self)
end

This allows you to avoid creating another closure function. However, it seems not very useful because hooks work only in one context (i.e., one script only and required lua files in this script).

• The tests from the repository show one failed test:
  DEBUG:SCRIPT: Pigeon tests end -------- [ PASSED: 20 FAILED: 1 ]

• [minor] The example cannot be run on Defold 1.4.2.

[Pawel]

Thank you so much for testing it! So on which Defold version were you testing? (For me it passes all tests)

I will be looking into all those issues after the jam, thank you for reporting them!

[Insality]

@Pawel

So on which Defold version were you testing?

Test on 1.4.5, just download & run

[Pawel]

Pigeon was updated to 1.1

GitHub - paweljarosz/pigeon at v1.1

The bugfix includes fix for pigeon.send() which was returning true, when no subscribers were subscribed to the given message. Now, in that situation:

• bugfix: pigeon.send() now correctly returns false, when no subscribers are subscribed to the given message and message is therfore not sent.
• The documentation is updated and the example project works in Defold 1.6.2.

Because of fixing this I also came back to this topic and are now trying to address the issues regarding the hook context and subscription mentioned by @Insality above

[tiberiuichim]

I’m trying to replace msg.post with pigeon.send. I have a problem with order of instantiation of game objects. Although the game is really simple, just two components, a board and a dial, when the board script posts, in its init(), the pigeon.send message, it seems to be too early, the dial game object hasn’t been initiated yet. The dial init() calls the pigeon.subscribe().

DEBUG:SCRIPT: sending time	8
DEBUG:SCRIPT: Pigeon: Failed to send message, id: set_time. Message_id is not subscribed to anything.	pigeon
DEBUG:SCRIPT: Pigeon: Successfully subscribed subscriber, id: 0	pigeon

The “sending time 8” is my own logging. I could move the send_time in the update() of the board script, or I could refactor things more, but I wonder if this is something that can be improved, or documented.

[Pawel]

Welcome to the Defold community!

Since you can’t set an order of init() calls if the objects are instantiated in one cycle*, such message, that I would need to send in initialization, I simply rather send from it’s own on_message, e.g:

board.script:

local pigeon = require "pigeon.pigeon"
local H = require "pigeon.hashed"

function init(self)
    msg.post("#", H.late_init)
end

function on_message(self, message_id, message)
    if message_id == H.late_init then
        -- do late-initialization here
        pigeon.send("to_other_subscriber")
    end
end

As of Pigeon 1.1 (check above) you also gets false from pigeon.send() when no subscribers are subscribed to given message, so something like this should wait to send the message to the one subscriber:

while(not pigeon.send("to_subscriber")) do end
-- or:
self.limit = 0
while(not pigeon.send("to_subscriber") and self.limit < 100 ) do self.limit = self.limit + 1 end

Although I do not recommend such loop that might become infinite. It’s better to write a controlled timer or at least add some kind of limiting counter as above, if you really don’t know when other subscriber will be subscribed.

Related:

GOs - Function init() - order of execution (SOLVED)

Dear, I’m using different Defold assets. Because of that I have run into some issues related to order of execution of used Game Objects in my collection. The question is if I have 3 objects A, B and C. How can I make sure that: function init() of A is first to be executed then function init() of B is Second and init() C is third I read the manual “core concept”, I couldn’t find something explaining the order of execution or how the engine decide which GO is first within the same GameCollec…

*From documentation:

The order in which game object component init() functions are called is unspecified. You should not assume that the engine initializes objects belonging to the same collection in a certain order.

Defold application lifecycle manual

[Chung_Xa]

Just look at the source code, it seems using msg.post to send messages. I have experienced with large data when using msg.post. Could you take care of it in Pigeon? By doing that, it would be great!

[Pawel]

If I understood you correctly, you want to send a lot of data, right? What’s your use case for passing huge data in message? It’s most probably better to pass a reference to a Lua table or reference to any resource, that is not copied in case of huge data.

msg.post allows to send up to 2 kilobytes of data, which is already pretty huge:

There is a hard limit to the message parameter table size. This limit is set to 2 kilobytes. There is currently no trivial way to figure out the exact memory size a table consumes but you can use collectgarbage("count") at before and after inserting the table to monitor memory use.

Message passing in Defold

[Chung_Xa]

Yes, you’re right. We shouldn’t send large data instead we send a reference of it. Here I mean it would be great if Pegion takes care of it for us so we won’t care about making references and getting data from the reference

[tiberiuichim]

Thanks for the warm welcome!

I’ve refactored my code using the late_init method. It was the only one needed in my game, as the other ones are chained from the main one. I have successfully replaced about 10 msg.post() in my game.

Thanks for creating this library! I’ll try to send a PR to include this bit of documentation.

[Chung_Xa]

I’ve just tried Pigeon and got this error. I’m not sure if it comes from Pigeon or Defold. Please help!

[Pawel]

It’s regarding what @insality described above - hooks are called in context of the sender. If you send the message (using pigeon.send()) from some other script that is not gui_script, you can’t access gui from there (nor you want to affect your own gui if you are sending like this from other gui_script).

Hooks are designed for something else - to “hook” up your function to the action you are triggering, e.g. whenever you send a message (for example, like hooks on git can perform some actions when you push the code, etc.). In my opinion, it’s the way to use them, but I get, that you are trying to connect subscription with certain action e.g. something like “on_received”. Let me know what you think about it, guys.

In your case, pigeon (using msg.post) messages are send to the subscribers and they should be handled in “on_message” - this way you ensure the context is this script. So simply, instead of writing a function(message_id, message) ... end - move this logic to on_message:

if message_id == hash("set_main_menu_context") then
   ...
end

[Daremo]

Hello everyone ,
I have just started using pigeon and its very powerful.
However when I tried to use the “Unsubscribe_all” function it did nothing.
I fixed my issue myself buy just looping the normal “unsubscribe” (by id) function, but I would like to know if the problem is in the library or did i used the function wrong?

[Pawel]

This is everything that unsubscribe_all is doing:

-- Unsubscribe all saved subscriptions.
-- @return	result		[boolean]	- true if unsubscribed succesfully, false otherwise.
function M.unsubscribe_all()
	for i,subscriber in ipairs(subscribers) do
		if not unsubscribe(subscriber.id) then
			return false
		end
	end
	return true
end

What is the issue in your code? Are there any subscribers subscribed before unsubscribe_all?
You can also debug the module or punt print() there, it’s only Lua

[Daremo]

Yes, I have subscribers and when I unsubscribe by id its all good. However the unsubscribe all seems to not do anything and all the subscribers still exists.
By the way I have tried to edit the module itself to test some things but it doesnt let me save changes, do you know how can I do that?

[Pawel]

@Daremo

but it doesnt let me save changes, do you know how can I do that?

Yes, to do that you need to copy it somewhere and do the changes on the copy The dependencies are by default read-only (I don’t know of any way of changing this and I think it’s actually good :D)