Look at my mouse (3D Screen to World)

[selimanac]

I need a little help. I’ll be glad if anyone can show me the right direction…

My problem is actually quite simple; follow the mouse position on 3D even when camera is rotated.

So I use screen to world calculation. It works great until I start to rotate the camera around and I’m not sure how to handle this situation cause I thought projection matrix is enough to handle and I see it is not(I might be wrong of course)

My screen to world is similar to Rendy’s, except I’m not using frustum.
There is no custom render.script, just rotating the camera using Orbit Camera

Minimal project is here: GitHub - selimanac/look-at-my-mouse

local function screen_to_world(self)
	local inv = vmath.inv(self.camera_projection * self.camera_view)

	-- Center
	local center_x = self.mouse_screen_position.x - ((DISPLAY_WIDTH / 2))
	local center_y = self.mouse_screen_position.y - ((DISPLAY_HEIGHT / 2))

	-- Near & Far plane
	local near_world_position = vmath.vector4(inv * vmath.vector4(center_x, center_y, -1, 1))
	local far_world_position = vmath.vector4(inv * vmath.vector4(center_x, center_y, 1, 1))

	near_world_position = near_world_position / near_world_position.w
	far_world_position = far_world_position / far_world_position.w

	local frustum_z = (self.mouse_screen_position.z - self.camera_nearZ) / (self.camera_farZ - self.camera_nearZ)
	local world_position = vmath.lerp(frustum_z, near_world_position, far_world_position)

	self.mouse_world_position.x = world_position.x
	self.mouse_world_position.y = world_position.y
	self.mouse_world_position.z = world_position.z
end

function update(self, dt)
	-- ROTATE TO MOUSE
	if self.mouse_screen_position then
		self.camera_projection = go.get(camera_id, "projection")
		self.camera_view = go.get(camera_id, "view")
		b_screen_to_world(self)
	end

	if self.mouse_world_position then
		self.hero_rotation_angle = math.atan2((self.mouse_world_position.x - self.hero_position.x), (self.mouse_world_position.y - self.hero_position.y)) - math.pi
		self.hero_rotation = vmath.quat_rotation_y(-self.hero_rotation_angle)
		go.set_rotation(self.hero_rotation, hero_id)
	end
end

So, I know this is not a easy task to ask and this is just a hobby project, but I’ll be glad to hear your suggestions.

[Dat_Indie]

Not the solution but random thoughts:

1. rotate the player while rotating the camera so you won’t “lose” the mouse cursor
2. at the end of the camera rotation call a function to rotate the player to face the cursor.

[Dragosha]

I don’t know how it works with these extensions, but with the old and well made Rendercam it’s simple:

local up 	= vmath.vector3(0, 1, 0)
local pivot = vmath.vector3(0, 8, 0)
...
 self.look_at = rendercam.screen_to_world_plane(action.screen_x, action.screen_y, up, pivot)
...

works with any camera angle because it doesn’t matter. We know the position of objects and the cursor in the world coordinate system.

Also I prefer euler.y instead quat for the hero rotation.

local angle = lume.angle(self.look_at.x, self.look_at.z, self.position.x, self.position.z) 
self.angle = -math.deg(angle) - 90 -- depends on origin model rotation
local old_angle = go.get(self.dude, "euler.y")
go.set(self.dude, "euler.y", math3d.lerp_angle(.92, self.angle, old_angle))

[selimanac]

@Dragosha

I don’t know how it works with these extensions, but with the old and well made Rendercam it’s simple:

Actually I’m not using any extensions, I don’t want to.
Orbitcam is just a basic default camera rotation script which I don’t want to spend time on it.
Rendercam is a good old big boy I’ll check screen_to_world_plane source. But I don’t want to touch the default(and new) render.script(no reason, just a challenge).

@Dragosha

Also I prefer euler.y instead quat for the hero rotation.

Any reason to prefer it? (easy, accurate…)

Thank you

[selimanac]

@Dat_Indie

• rotate the player while rotating the camera so you won’t “lose” the mouse cursor
• at the end of the camera rotation call a function to rotate the player to face the cursor.

Might help, I’ll try.

Thank you

[ross.grams]

Hmm… The first problem is you need to convert the screen position into -1 to +1 coordinates. In other words: you should be dividing by the window size in there somewhere.

Next, what is the “screen” Z position you are using? The way you are using it in the code, it’s a distance from the camera in the view direction, but what you seem to want for your actual game is a point on a 3D plane, so those two things don’t match up.

Otherwise your math looks right.

[Edit:] Rendercam needed to modify the render script to manage its own view & projection matrices, among other things. With the very recent updates to the built-in camera, you can now get the matrices from the camera component (as I see you are doing), so you shouldn’t need to modify the render script at all. Of course these matrices will still be out of date if you moved or rotated the camera that frame (also an issue with Rendercam).

[selimanac]

@ross.grams

Hmm… The first problem is you need to convert the screen position into -1 to +1 coordinates. In other words: you should be dividing by the window size in there somewhere.

Next, what is the “screen” Z position you are using? The way you are using it in the code, it’s a distance from the camera in the view direction, but what you seem to want for your actual game is a point on a 3D plane, so those two things don’t match up.

Thank you very much @ross.grams this is very helpful to me.

@ross.grams

so you shouldn’t need to modify the render script at all. Of course these matrices will still be out of date if you moved or rotated the camera that frame (also an issue with Rendercam).

Yes, I manage to make screen_to_world_plane work with build-in camera and without modify the render.script. It is better now. But, I would like to try to fix my initial code by your suggestions (for learning purposes)