Is there any nice way of transforming a screen coordinate to world space?

Questions
#22

Hold your horses. You are overreacting. If you would have asked this question here then we would have presented you with two camera extensions, both professional quality in terms of stability and performance. I would also have shared this extension specifically aimed at input handling: https://github.com/britzl/defold-input and this basic example: https://github.com/britzl/publicexamples/tree/master/examples/click_game_object

We should perhaps mention these in one of our manuals, but for now we recommend them here. Ping @sicher

4 Likes
#23

Yes. I wonder what would be a good place to put this information. Maybe a FAQ entry and something in the input manul perhaps?

1 Like
#24

Please spend at least a little time to read up on the product and ask questions before jumping to conclusions. Also, we try to keep the tone civil here.

3 Likes
#25

First I would like to apologize for the tone of my first post. I donā€™t usually go off like that and it was at the end of a long day where I was trying to do something that should have been simple.

Second I always try to find if my question has been asked already before posting it. It had and that thread led me to this one (sorry I lost track of the original thread).

Third this is not the only issue I have had with the engine. One of the first was when I found out there was no simple way to lock a mobile game into landscape orientation when almost every mobile game I play is played in landscape mode.

Last I would like to give a bit of my backstory. I have an autistic son, a 4-year old son, and a 2-year old daughter who all really loved a certain balloon popping game on their kindles. The problem was that it was so full of adds they would often click them and get frustrated wanting the game back. I have a degree in game development that I donā€™t use for work and I decided to try my hand at remaking the game so I could add things as they got older.

I originally made the game using phaser-ce and put it on a webserver. Then the issue was they would click on the address bar and get frustrated. So I googled ā€˜2d mobile game engineā€™ and this was one of the top picks. At first it seemed simple enough. I caught the line ā€˜this is technically a 3d engine optimized for 2dā€™ in the documentation and that didnā€™t bother me.

I first started being frustrated with the editor. No code folding, not being able to expand a node if it selected, funky keyboard shortcuts for my Dvorak keyboard (using scancodes instead of actual keys for the keyboard shortcuts). I grit my teeth and dealt with it. I worked through most of the tutorials to learn the ins and outs of the engine and then took a stab at building my balloon pop game. I didnā€™t need a loading screen or even a UI as this was going to be simply for personal use.

It started off going well. A little hiccup with the landscape orientation I ignored. I was able to add the background, and figured out how to use a tilemap for the balloons (which is stored as a single image). I figured out how to use the physics system to give the balloons a bit of gravity pulling them up. I figured out how to destroy the balloons when they leave the screen. Then I got to the tapping part and got stymied (yes I said stymied). That led to this post as I had spent several days learning the engine and tools to find out something as simple as tapping on a sprite couldnā€™t be handled by the engine. If there are plugins they should be mentioned in the documentation.

Since my initial post I have rediscovered love2d which has come out of beta since I last looked at it. It shows great promise as it took me less than a day to get to the same point in development as I got with defold in 3 days of work.

I wish you luck with defold. I think the engine needs a lot of work though and trying to gear a 3d engine for 2d games seems a bit more cumbersome than I would want to deal with.

3 Likes
#26

Yes, thereā€™s a very simple way. Set width and height in game.project to a landscape ratio (ie wider than the screen is high) and make sure that Dynamic Orientation is unchecked.

Thereā€™s currently a bug that if you leave the default values of 960x640 they will actually not be used and the game will end up in portrait mode anyway. This will be fixed in Defold 1.2.136 (in two weeks).

It is not as simple as it sounds. You as a developer have full control over the rendering pipeline. It means that you can do absolutely anything with how the game is rendered. You can render the entire game in the top left corner of the screen or zoomed in our inverted or anything else. This makes it impossible for the input system to know how to translate screen coordinates to world coordinates and by that also know exactly where on the screen a game object is rendered.

Some engines may have solved this by limiting the rendering or through an advanced camera system that takes care of input translation. We decided to not solve it for the user and instead let the user pick a solution that works for them. Two solutions are RenderCam and Defold-Orthographic, both available from our Asset Portal.

This is part of the problem. The second part is how to once you have a world coordinate know if that coordinate is touching a sprite. Some engines solve this for you. We donā€™t, well at least partially. For gui nodes there is gui.pick_node(node, x,y). For game objects there are a number of different solutions. One that is provided by the engine is through the use of collision objects. One collision object component together with the sprite and one collision component following the mouse or touch. Quite easy to set up.

Our ambition with Defold is not to solve all problems for our developers. Instead we try to provide a set of APIs and tools that allows developers to solve the problems themselves in a way that bests suits their game and their target platform(s). This philosophy doesnā€™t suit everyone, and for those that donā€™t like it thereā€™s plenty of other game engines to use. Good luck with Love2D!

12 Likes
#27

I just want to add that after a whole day wasted of trying to solve this problem without using the extremely complicated ā€˜add onsā€™ suggested here, Iā€™m also giving up on defold. It is completely insane that a 2D-first engine has no built-in solution for getting world x and y coordinates in the on_input lifecycle, it only provides screen x,y and some useless ā€œx,yā€ which will break as soon as the window is resized in any way. It also appears zoom messes with it, making this even more of a nightmare for pixel art games. Just mind boggling how the simple stuff is not addressed at all.

Perhaps all of this could be solved in a simpler way using a GUI overlay of buttons, it seems overly complicated and of course there is no hint of this in the manual, or this years old post. Everything in defold seems to be done from the perspective of making things unnecessarily vague. Too much freedom means your software is not user friendly. Make a clear path of the best way to handle common problems, and leave a ā€˜free wayā€™ to do things for those that really want to. Donā€™t leave people in the dark, itā€™s just a sign of you donā€™t actually have solutions to common problems.

The process for installing those third party plugins is about the most hacky non-step by step process Iā€™ve ever seen. You pretty much have to open the examples in the defold editor and scrutinize the code to figure out how they did things. For example there is a require statement in the orthographic add-on script which is nowhere to be found on the github how-to. Thereā€™s also hidden ā€˜problemsā€™ like adding library dependencies doesnā€™t seem to always work until you re-launch defold. WTF.

#28

I am sorry that you had all these problems in converting screen to world coordinates.

Are you using any kind of camera? are you using a custom render script? In any case the math you need should be very very simple. Could you perhaps let us know your context?

Ciao!

#29

Iā€™m sorry to hear that.

Defold has been labeled 2D focused, but it is a 3D game engine when it comes to how the game world is represented and rendered.

Would you mind sharing what specifically wasnā€™t clear in the instructions so that we/I can improve them?

Iā€™m happy to add it if you can show what you mean.

When you add a dependency in an open project you also need to select ā€œProject->Fetch Dependenciesā€. This is documented in the official manual:

I completely agree with you that it is bad that we donā€™t have an official and simple to use solution for this problem. I think weā€™ve been reluctant to add something which doesnā€™t work for multiple cameras and the different default projections in the render script. You can expect this to change early this year. Me and @AGulev have discussed this exact problem extensively and we have a relatively good idea of how we want to approach this problem. We have recently taken some steps in the right direction:

  1. The camera component now exposes the projection and view. These are needed when you wish to convert coordinates
  2. The render script has been refactored.

Next step is to add a unified API function to convert screen to world coordinates given a camera or a default projection from the render script.

Until we have this in place you can probably be helped by this sample project which I put together. It shows how to get the view and projection from the camera and do the conversion:

The relevant function using the camera projections and view to do the conversion:

The sample project will track and convert the mouse position to a world position and let you move a tank on a large map while a camera follows the tank.

7 Likes
#30

I obviously donā€™t know enough about this problem but I would think that the fix should be that actions in the on_input lifecycle should have action.world_x and action.world_y, which is figured out based on the current active camera (if there is one), or figured out without a custom camera. If this only applies to 2D games, then call it world_x_2D and world_y_2D or something.

I will take a look at the sample-screen-to-world-coordinates project later, the only way I got the orthographic addon to work was to add it as a project, delete everything but the allfeatures collection, and kind of sift through and find the things that werenā€™t needed like the test folder, reconfigure all the game.project settings and start from scratch. There is no clear step-by-step process on how to set up the scene, collections, cameras, scripts, etc. to actually get a world_x and world_y. In native defold you hook into the on_input action.x and action.y, in the addon it seems to somehow hook up like this:

local camera = require "orthographic.camera"

local CAMERA_ID = hash("/camera")

function init(self)
	self.crosshair = vmath.vector3()
	msg.post(".", "acquire_input_focus")
end

function update(self, dt)
	local cursor_pos = camera.screen_to_world(CAMERA_ID, self.crosshair)
	cursor_pos = vmath.lerp(0.5, go.get_position(), cursor_pos)
	go.set_position(cursor_pos)
	label.set_text("#position", ("%.2f x %.2f"):format(cursor_pos.x, cursor_pos.y))
end

function on_input(self, action_id, action)
	self.crosshair.x = action.x
	self.crosshair.y = action.y
end

Why it takes a crosshair vector, how this crosshair pertains to the cursor position is a mystery, why it goes through the update function rather than on_input is a mystery, why the crosshair is again being set back to the old ā€˜brokenā€™ action.x and action.y in the on_input function is another mystery - entire thing looks like black magic.

#31

How can this be a mystery?

The crosshair vector is the mouse screen position. It is updated every on_input:

function on_input(self, action_id, action)
	self.crosshair.x = action.x
	self.crosshair.y = action.y
end

And then used in update to get a world position:

local cursor_pos = camera.screen_to_world(CAMERA_ID, self.crosshair)

In this case there is a linear interpolation of the cursor, but that is only there for the visuals. You could updated the position in on_input() to move it instantly.

The on_input function is called every time you get an input event such as a touch or mouse event. And every time self.crosshair is set to the screen coordinates of the input event. And every frame these screen coordinates are converted to world coordinates in update and the position is updated.

4 Likes
#32

That makes sense, I think the mystery came from not knowing when update runs vs on_input (which comes first) and so on, and how they chain together. Reading top to bottom I was mistakenly thinking after init and update, you were setting the crosshair to be action.x and action.y.

#33

The application life-cycle and the engine update loop is documented here: Defold application lifecycle manual

2 Likes