Gui.pick_node doesn't work with custom GUI matrix (SOLVED)

Questions
1

Hello. I created custom matrix for my ui.

render.set_view(vmath.matrix4())
render.set_projection(vmath.matrix4_orthographic(0, ui_projection_width, 0, ui_projection_height, -1, 1))
render.set_viewport(ui_viewport_x, ui_viewport_y, ui_viewport_width, ui_viewport_height)

and then i use following function to convert screen coordinates to UI coordinates

function update_screen_position(screen_position_x, screen_position_y)
	screen_ui_position.x = (screen_position_x - ui_viewport_x) / ui_screen_ratio.x
	screen_ui_position.y = (screen_position_y - ui_viewport_y) / ui_screen_ratio.y	
end

I have base UI size 800/600 and somewhere in render script i calculate ui_screen_ratio. so my function works well on desktop when i call it and handle in gui script like in code below

function on_input(self, action_id, action)		
	if action_id == hash("touch") and action.pressed then
		pprint(action)
		update_screen_position(action.screen_x, action.screen_y)
	    if gui.pick_node(self.ui.buttons.restart, screen_ui_position.x, screen_ui_position.y) then	    	
	    	msg.post("default:/board#controller", RESTART_MESSAGE, {})
    	elseif gui.pick_node(self.ui.buttons.help, screen_ui_position.x, screen_ui_position.y) then
   			change_state(self, STATE_HELP)
   		elseif gui.pick_node(self.ui.buttons.twitter, screen_ui_position.x, screen_ui_position.y) then
   			open_twitter(self)		
   		elseif gui.pick_node(self.ui.buttons.help_back, screen_ui_position.x, screen_ui_position.y) then
   			change_state(self, STATE_BASE) 		
		end
	end
end

problem happens when i come to android. action.x and action.y contains another coordinates not the same as screen pixel positions. and even if i use action.screen_x and action.screen_y sims like gui.pick_node doesn’t handle rects with calculated screen coordinates

1 Like
2

Finally using imperial approach i found the solution =) and i’m still confused.
to hit button at (100, 50) position. with default screen size (800;600) I must provide vector(100,50) coordinates, but for bigger screen size like (1234;600) this coordinates will be (59,50)
my function to transform screen coordinates to ui coordinates looks like

function update_screen_position(screen_position_x, screen_position_y)
	
	local relative_x = (screen_position_x - ui_viewport_x) / ui_screen_ratio.x  
	local relative_y = (screen_position_y - ui_viewport_y) / ui_screen_ratio.y			
		
	action_ui_position.x = relative_x * base_screen_width / width  
	action_ui_position.y = relative_y * base_screen_height / height	
end

and i call it with screen coordinates like this

update_screen_position(action.screen_x, action.screen_y)

not action.x or action.y

2 Likes
3

I’m again here with the same issue (and the same already solved) but today i want to provide more details. so if someone will want to change GUI matrix he will know how to solve input issue more quickly.

First of all let me show you this diagram and describe some terms

DefoldGUI.PNG1347×861 5.47 KB

  1. In Defold we have Layout settings where we define what size of our gui. On the diagram GUI layer is A’B’C’D’ square and the zero point is (0:0).
  2. By default DEFOLD just project A’B’C’D to the ABCD square on the screen. so no matter what is your device screen size this ui will be just stretched by default to the ABCD square and it’s quite ok because you can stick some nodes and then just change Adjust Mode if you want to keep original node aspect or so on.
  3. In my case I want to stick the whole ui to the center of the screen. In the center i have a board and right on the side of the game board i have ui.
  4. So what I’ve done is just setted up custom projection matrix for the whole ui. this is what’s usually camera means in another engines like unity (you setup the size and aspect). here is my code for landscape screen
        local actual_window_aspect = render.get_window_width() / render.get_window_height()
        local camera_width = gui.get_height() * atual_window_aspect
        local camera_left = -(camera_width - gui.get_width()) / 2
        local camera_right = camera_left + camera_width
        self.ui_matrix = vmath.matrix4_orthographic(camera_left, camera_right, 0, render.get_window_height(), -100, 100)

Here we let our camera know left right bottom and top border which we want to show on our screen (it could be E’F’G’H’ on that diagramm but i didn’t draw that). so these data is located on the same coordinate system as your nodes in your ui
5. When you setup your camera no mater how you will stretch your window if you will have landscape mode all elements will be stored on the same position relate to center.

No you will see that your input doesn’t work corectly if your window size changed. and you need to call gui.pick_node with calculated coordinates. to do that I use following code

function update_screen_position(screen_position_x, screen_position_y)
    screen_ui_position.x = (screen_position_x - screen_gui.left) / screen_gui.width * gui.get_width() * gui.get_width() / window_size.x
    screen_ui_position.y = (screen_position_y - screen_gui.bottom) / screen_gui.height * gui.get_height() * gui.get_height() / window_size.y
end

so in any on_input method i call this function and then use screen_ui_position with gui.pick_node

  1. screen_position is the action.screen_x and action.screen_y see here
  2. screen_gui.left is the C point on the diagram above
  3. screen_gui.width is the C->D segment
  4. gui.get_width() is from defold API
  5. window_size.x is the size of the screen which you can get as render.get_window_width()

Let me show you some examples.

  1. we have a button with following properties x=20, y=90, width=180, height=50
  2. Gui layout width = 800, height = 600
  3. With game screen size 800x600 gui.pick_node return true if 20<x<(180+20) and 90<y<(90+50) quite straight forward and you don’t even use the function above and take x and y from action
  4. With screen size 1600x600 y stuff works the same but how do you know which X coordinates will be acepted?
    the right answer is 10<x<100 this happens because action.x normal diapason (if you don’t change) the matrix is 0<x<gui.get_width() but our screen become much wider and ABCD square become relatively smaller. and then i think someone from defold team will provide more math formulas why this happens =). actually i’m still a bit confused about all of that.

pleas let me know if you want me to fix some errors in the text above. or fill free to edit the comment if you have r/w access to another users comments =).

4 Likes