Get mobile keyboard with html5 (DEF-3503)

Pkeod · September 15, 2018, 7:36am

Someone motivated could make a Native Extension for this.

bndibx · September 18, 2018, 9:25am

How would the extension? Would it be a html5 extension that can all up the android keyboard? Any pointers would be helpful…
I tried modding the example android extension to call up a keyboard (InputMethodManager) but it still didn’t work on mobile html5

Pkeod · September 18, 2018, 9:46am

For mobile keyboard you have to create an input field and then give it focus, then get the text entered to it. If you search like “html5 game mobile keyboard input” you can find examples. Then you want to have a function to deal with this text input in your game and close the input focus when entered. You probably don’t even need a NE for this it might be more easily doable with html5.run() and then polling a listener on update while waiting for input. If you’re not experience with JS this might be harder for you but try to look up examples and see if you can get it working on your own first.

bndibx · September 20, 2018, 7:59am

So for the last two days I’ve been trying to get the keyboard to show up but it simply won’t popup…Here’s my pure javascript code. Kindly assist.

If I unhide the input field, I can see the field gets focus but no keyboard pops out. Tested on both Firefox and Chrome Android

EDIT: I got the keyboard to show by putting my html5.run() in on_input, also remove the hidden = true line, you can hide the input_field with inline css.

Now I just have to figure out how to get each key clicked back to my game.

@Pkeod, You said I should poll a listener ? Can you assist me with this part?

You should be able to drop the code in a defold project and have it run.


js_data = [[
	var input = document.createElement('input');
	input.type = "text";
	input.name = "test";
	input.setAttribute('id', "foo");
	input.addEventListener('click', isClicked);
	document.body.appendChild(input);
        document.getElementById("foo").hidden = true;

	function isClicked(){
		this.focus();
	};
	document.getElementById('foo').click();

	-- window.onload = function () { 
	--	document.getElementById('foo') && 
		--document.getElementById('foo').focus(); };
    ]]

	html5.run(js_data)

britzl · September 20, 2018, 9:33am

I’ve created a ticket to track a solution where this is handled by the engine: DEF-3503

I can’t make any promises on when this would be supported natively by Defold though.

decoded · September 20, 2018, 2:41pm

You should be able to set up a timer that will html5_run() javascript to return the text at document.getElementById(“foo”).value

Interested to see how this turns out. I need an HTML5 mobile keyboard too.

bndibx · September 25, 2018, 1:41pm

Doesn’t work. Every key pressed on mobile keyboard currently responds with keyCode 229 except some number pads on some keyboards. Gboard, for example gives only keyCode 229 while the default Android keyboard only number keys respond correctly. It’s so confusing…

decoded · September 25, 2018, 5:37pm

Hmm. We shouldn’t be getting any sort of keycodes.

I tried this on Android using your code and adding the timer, and it seems to work for me.

local function get_text()
	local js_data = [[
		function get_text_value(){
			var value = document.getElementById("foo").value;
			return value;
		};
		get_text_value();
	]]
	local text_value = html5.run(js_data)
	--print value to console
	pprint(text_value) 
	--set value to a GUI text node (requires gui node setup)
	--gui.set_text(gui.get_node("text_node"), text_value)
end

function init(self)
	msg.post(".", "acquire_input_focus")
	
	local js_data = [[
	var input = document.createElement('input');
	input.type = "text";
	input.name = "test";
	input.setAttribute('id', "foo");
	input.addEventListener('click', isClicked);
	document.body.appendChild(input);
	function isClicked(){
		this.focus();
	};
	document.getElementById('foo').click();
	]]

	html5.run(js_data)
	timer.delay(0.5, true, get_text)
end

Note this is just example - finer control of the timer should probably be used in production.
Also, thanks for the neat little way to include the JS, I was originally using an index.html template

Dragosha · September 25, 2018, 6:12pm

This is first reason why i made retro ingame keyboard for player’s nickname instead using of native keyboard:

The second reason is control on non-latin alphabet. Don’t forget that user can try to input someone like “моё замечательное имя” and your UI should not break from this.

decoded · September 25, 2018, 6:34pm

That seems like the best way to go. It’s obviously a lot more work, but the results look great and have many benefits.

Also to confirm,
“моё замечательное имя” in the JavaScript example results in GUI value like this:

I think maybe fonts could be configured to display it correctly though, no?

bndibx · December 7, 2018, 1:34pm

Any updates on this? Was hoping WASM changed things for mobile but it still doesn’t work.

I tried the on screen keyboard but unfortunately it’s been our most voiced concerns from QA, since we require user input a couple of times and game is mostly played on mobile browsers.

I tried adding a Custom Virtual Keyboard that inserts user input to a hidden field in index.html then polling that field every 0 seconds via html5.run() but it’s buggy and simply isn’t a replacement for the native mobile keyboard.

Kindly assist.

britzl · December 10, 2018, 7:39am

We don’t have an update for this I’m afraid. I did however play around with this a bit during the weekend. On HTML5 it’s possible to at run-time create an input field and give that focus to bring up the native virtual keyboard. You can then poll for input in the text field and forward the input to your Lua code.

CODE: https://github.com/britzl/defold-input/blob/master/in/keyboard.lua
DEMO: https://britzl.github.io/Defold-Input/ (click on the Keyboard button to test)

Note: I wasn’t able to get the native keyboard to show when the canvas was fullscreen though. Not sure if that is possible to be honest.

bndibx · December 19, 2018, 8:07am

@britzl Thanks for the code. It works great on Android but doesn’t seem to respond on iOS most of the time. Any ideas?

britzl · December 19, 2018, 8:10am

I only tested on Android. The keyboard.lua modul polls the keyboard very frequently to make it feel responsive, but I’m thinking that it does so too frequently:

github.com

britzl/defold-input/blob/master/in/keyboard.lua#L77


		return
	end


	show_keyboard(keyboard_type)
	if timer_handle then
		--log("stopping timer")
		timer.cancel(timer_handle)
		timer_handle = nil
	end


	timer_handle = timer.delay(0.05, true, function()
		if not has_focus() and timer_handle then
			--log("timer lost focus")
			timer.cancel(timer_handle)
			timer_handle = nil
		end
		local text = html5.run(ELEMENT .. ".value")
		if text ~= current_text then
			current_text = text
			msg.post(".", M.KEYBOARD_INPUT, { text = text })
		end

Let me give it a try on an iOS device.

britzl · December 19, 2018, 10:53am

Jeez. Safari prevents DOM elements from programmatic focus (which is what I do to show the keyboard) when the focus() call happens outside of an input event (which is the case when doing it from Defold). I’m currently stuck unless someone has a clever idea…

decoded · March 20, 2019, 7:38pm

Something like,

in the Asset Portal would be mighty nice.

totebo · July 11, 2019, 10:56am

I’m guessing using the virtual keyboard in HTML5 on iOS is still a brick wall?

britzl · July 11, 2019, 7:02pm

I think so. I haven’t googled for a solution since this was discussed last.

dlannan · August 17, 2021, 11:34pm

Hi, I know this is an old thread, but I thought Id share something I managed to get working. The code shown below does a few different things, so I will explain after the code.

	self.platform_info = sys.get_sys_info()
-- The system OS name: "Darwin", "Linux", "Windows", "HTML5", "Android" or "iPhone OS"
if self.platform_info.system_name == "HTML5" then
	local html_data = [[
	var html = "<div style='position:absolute;left:0px;top:0px;z-index:-1;'>";
	html = html + "<input id='hiddenInput' type='text' name='hiddenInput' /></div>";
	document.getElementById("canvas-container").insertAdjacentHTML("afterend", html);
]]
html5.run(html_data)
		
local js_data = [[
	var script = document.createElement("script");
	var funcscr = "var inputcurr = '';\n";
	funcscr = funcscr + "var inputdata = {};\n";
	funcscr = funcscr + "var winsize = [window.width,window.height];\n";

	funcscr = funcscr + "window.addEventListener('resize', windowSizeChanged);\n";
	funcscr = funcscr + "window.addEventListener('load', function() {\n";
	funcscr = funcscr + "  winsize = [window.width,window.height];\n";
	funcscr = funcscr + "})\n";

	funcscr = funcscr + "function windowSizeChanged() {\n";
	funcscr = funcscr + "  window.resizeTo(winsize[0],winsize[1]);\n";
	funcscr = funcscr + "}\n";
	
	funcscr = funcscr + "const inputtext = document.querySelector('#hiddenInput');\n";
	funcscr = funcscr + "inputtext.addEventListener('keydown', (e) => {\n";
	funcscr = funcscr + "  var key = e.key;\n";
	funcscr = funcscr + "});\n";

	funcscr = funcscr + "function clearFocus(){ \n ";
	funcscr = funcscr + "  inputcurr = 'none';\n";
	funcscr = funcscr + "  var canvas = document.getElementById('canvas');\n";
	funcscr = funcscr + "  canvas.focus();\n";
	funcscr = funcscr + "}\n";
	
	funcscr = funcscr + "function clearInput(id, curr){ \n ";
	funcscr = funcscr + "  inputdata[id] = '';\n";
	funcscr = funcscr + "  inputcurr = id;\n";
	funcscr = funcscr + "  document.getElementById('hiddenInput').value = curr;\n";
	funcscr = funcscr + "  var input = document.getElementById('hiddenInput');\n";
	funcscr = funcscr + "  input.focus();\n";
	funcscr = funcscr + "}\n";
	
	funcscr = funcscr + "function getInputText(id){ \n ";
	funcscr = funcscr + "  var input = document.getElementById('hiddenInput');\n";
	funcscr = funcscr + "  input.focus();\n";
	funcscr = funcscr + "  inputdata[id] = input.value;\n";
	funcscr = funcscr + "  return inputdata[id];\n";
	funcscr = funcscr + "}\n";

	script.innerHTML = funcscr;
	document.body.appendChild(script);
	]]

	html5.run(js_data)
end

There are two sections of code. The first adds some html to the canvas container to provide a hidden input text box as others have described above.

The second set of code injects a resize handler, keydown handler and a number of functions to be used in the Defold script.

Within your script, when you want to start getting the text from the input box use:

html5.run( "clearInput('"..myid.."', 'starting text')" )

This will insert “starting text” into the hidden text box and create a mapped response using myid defold variable - note the myid must be some valid string. Whenever you want to use the text input of that input you can use:

local value = html5.run("getInputText('"..myid.."')")

The clearFocus function is used to hide away the keyboard.
One additional thing you may need to to is to disable the “resize” handler in the default js-web template. This is because if you have the keyboard popup the game will attempt to resize, which is not really useful in these cases. Heres where I commented out the handler in my js-web template:

	resize_game_canvas();
//window.addEventListener('resize', resize_game_canvas, false);    <----------------------
window.addEventListener('orientationchange', resize_game_canvas, false);
window.addEventListener('focus', resize_game_canvas, false);
</script>

Hope this helps. Im glad to have a working solution for android. Will need to test on IOS.

dlannan · September 5, 2021, 12:48am

Ok. It looks like the above has issues running on Safari. I am investigating and hope to have a fix soon.