Need help with interacting in 3d

Questions
#1

Hello all,

my model is now done apart from proper lights. I use camera projection, the camera is attached to the player go, it can be rotated and also be operated with the mouse wheel and the arrow keys. But I am not able to get the player to click on objects and interact with them.
For days now, I have looked at every thread I could find here, look at every example mentioned but cannot get my head around it and I am now ready to abandon the project.
As a last resort, I now need to ask for your help: maybe someone has a small sample project somewhere or a code snipped that could help me out? This would be so wonderful!

#2

What kind of objects are we talking about? 3D objects in the world? How do you detect clicks? I suggest using physics shapes and ray casts with physics set to 3D in game.project.

1 Like
#3

Hello @britzl,

thank you very much for your reply!
You know my 2d museum project, and I got very ambitious and now want to build a 3d museum. Too ambitious, it seems. Well, the museum is done and dusted, the paintings on the wall. They are 3d objects with a collision shape. And if the visitor clicks, I would like to give them some info about what they are looking at or do some other stuff, I don’t know yet.

Exactly, no idea! I just know at the moment when I bumped into an object.
I added the code found in the manual (“Converting mouse to world coordinates”), but it seems to return wrong values.

Ah, yes, I came across them :smiley: - don’t know yet what to do with them. This could be a solution for me, I will try out right now - thank you!

2 Likes
#4

So basically, your question is: How do I detect the 3D object that the camera is looking at?

In 2D, to get click collision you just have to check for objects at a point (x, y). In 3D, you have an extra axis: depth (or distance-from-the-camera). For any (x, y) point on the screen, there could be an object that is very close, or an object that is very far away. So instead of just checking for collision at a single point, you need to check along a 3D line going straight away from the camera (and get the closest object).

A simple way to do this: cast a ray (with physics.raycast()) from the camera position to a point straight in front of the camera, some distance away (whatever maximum click distance you want)

If your mouse cursor can move around on the screen and you want to detect what you click on, then it’s a bit more complicated. Instead of the ray going straight away from the camera, it’ll be at an angle depending on where on the screen you clicked. I don’t know that there’s an easy, drop-in way to do this, but if you’re using Rendercam then it’s done for you with rendercam.screen_to_world_ray().

You’ll probably want to draw some debugging lines to visualize things while you’re sorting this out.

6 Likes
#5

Hello @ross.grams,

thank you! I am just looking at ray casts like @britzl just suggested. Already, my walls are detected, cool. I did study your RenderCam example this morning and destroyed it in the process :laughing:

So sad :grinning: - it is indeed quite complicated and my debug print statements for now not what I wish them to be.

1 Like
#6

I think the best way forward is to use RenderCam as it has the things you need. Or maybe you can build on the First Person Controller by @Ivan_Lytkin?

Or perhaps @aglitchman will release some additional helpers for that kind of thing? Already released: Pointer Lock and he’s teased with this: https://twitter.com/aglitchman/status/1487565479174619141

2 Likes
#7

Hi there again,

Oh yes, I have studied this great example (that also uses Pointer Lock) and also @Pkeod’s “CameraCon” thread. Both enabled me to learn how to set up the movement of the camera within my 3d world - thank’s guys! Alas, there is no interaction with game objects going on in these examples and that’s where I got stuck.
I’ll have another look at RenderCam and ray casts as you and @ross.grams suggested, so thank you both.
While working, I stumbled across this, probably not worthy of a thread:
When copying or moving collision objects, the shapes lose their, well, shape:

image
image823×446 26.2 KB

Closing and restarting Defold brings them back to normal (Mac mini M1, Monterey 12.1)

#8

Do you want mechanics as I did in Driller? I.e. when you can touch a virtual item.

Just do a raycast from the centre of the screen = from your camera.

Yes, it’s a known issue. Take a look: https://github.com/defold/defold/issues/5049.

2 Likes
#9

Hello @aglitchman,

thank you very much! I had downloaded Driller - very cool! I will once more study the code and get my rays to hit the right objects. Still off target at the moment but when I get them to hit what I want, then that’s the solution to my scenario.
So, thank you all here for helping me out by suggesting to use raycasting. I am sure I will get there eventually.

2 Likes
#10

Hello @ross.grams,

I followed your advice and used rendercam.screen_to_world_ray().
When I store it in a var and print it out, only the start point prints.

local test = rendercam.screen_to_world_ray(action.x, action.y, false)
-- prints only the start point - how to get at the end point?
print(test)

So, I lay my hands on your script (sorry!) and split the function in two parts:

local start_point = rendercam.screen_to_world_ray_start(action.x, action.y, false)
--print(start_point)
local end_point = rendercam.screen_to_world_ray_end(action.x, action.y, false)
--print(end_point)

This works great and the ray now detects my targets - brilliant!
But may I ask you how to properly use the function as you intended (get at start and end)?

2 Likes
#11

This function returns two points:

local start_point, end_point = rendercam.screen_to_world_ray(action.x, action.y, false)
4 Likes
#12

Hi @britzl,

of course, that’s how it is done! I keep forgetting about it :flushed: Thank you so much.
Oh, and once more to all of you here that gave me the all important advice on how to detect clicked objects in a 3d world to be able to interact with them.

3 Likes